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1 Introduction and aims 



1 .1 About the research 

The Government’s proposals for reforming children’s services were set out in 
the 2003 Green Paper Every Child Matters (HM Treasury, 2003). The proposals 
aimed to combine the development of an overall framework for universal chil- 
dren’s services with the need for targeted services to protect vulnerable children. 
The Green Paper introduced five outcomes for children’s services, proposed by 
children and young people through extensive consultation, as being of key 
importance during childhood and adult life: 

• being healthy 

• staying safe 

• enjoying and achieving 

• making a positive contribution 

• achieving economic well-being. 

Together, the five outcomes encompass the overall aims for every child and 
young person. Following Every Child Matters (ECM) there was wide consulta- 
tion with all the key agencies and with children and families, which led to the 
publication of Every Child Matters: Next Steps (DfES, 2004). The Children Act 
2004 (England and Wales Statutes, 2004) then made some of the ECM propos- 
als statutory. Every Child Matters: Change for Children (HM Government, 
2004) was published following The Children Act 2004 (England and Wales. 
Statutes, 2004) and set out the national outcomes framework for change in local 
authorities (LAs) and their partner organisations. This included 26 aims related 
to the five outcomes for children and young people, as well as the priority 
national targets and other indicators. 

Alongside the introduction of the changes at LA level were changes to the 
inspections of children’s services. The new inspections aim to be more out- 
come-focused and involve both the Office for Standards in Education (Ofsted) 
and the Commission for Social Care Inspection (CSCI) conducting joint Annual 
Performance Assessments (APAs) and Joint Area Reviews (JARs). The inspec- 
tion criteria also form part of the ECM outcomes framework. In line with the 
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new outcomes framework, the 150 LAs in England require a new tool for 
benchmarking progress against the ECM outcomes. 

Statistical neighbour models provide one method for benchmarking progress. 
For each LA these models designate a number of other LAs deemed to have 
similar characteristics. These designated LAs are known as statistical neigh- 
bours. Any LA may compare its performance (as measured by various 
indicators) against its statistical neighbours to provide an initial guide as to 
whether their performance is above or below the level that might be expected. 

Currently, Ofsted provide a statistical neighbour model based on census data 
and the Institute of Public Finance comparator councils provide a model based 
on deprivation and demography data. But there is now a need for a new model 
based on the five ECM outcomes, which embraces the key elements of the 
existing models and provides LAs and their partner agencies with a tool for 
assessing and comparing their performance with their statistical neighbours. 
Ideally this new model would supersede the existing models within the context 
of children’s services and provide a single starting point for benchmarking per- 
formance. 

In April 2006, the DfES commissioned the NFER to conduct an independent 
external review in order to develop a single ‘statistical neighbour’ model. This 
single model aimed to combine the key elements of the different models cur- 
rently available and be relevant to children and young people’s services, to 
support LAs with the ECM agenda. 

The new model, described in this report, was developed to ensure that it is 
appropriate for use in benchmarking across the range of indicators that are 
defined in the ECM Outcomes framework. This necessary focus for the model 
is distinct from previously developed methodologies and provided a guide for 
the decisions that determine how statistical neighbours are defined. 



1 .2 Aims of research 

The research aims to deliver a single statistical neighbour model by combining, 
and enhancing, the key elements of the different models currently available 
which are relevant to children and young people’s services. The specific objec- 
tives of the research were to: 
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• develop a robust model of statistical neighbours for all 150 LAs in England 

• provide reasoned criteria for the decisions made in the development of the 
statistical neighbours model 

• develop an easy to use tool in Excel format for displaying individual LAs 
with their statistical neighbours, as well as all 150 groups in one place for 
ease of comparative analysis. 

1 .3 Structure of report 

The research was broken down into two main areas: consultation with stake- 
holders (via an online bulletin board and telephone interviews); and technical 
research relating to grouping LAs into statistical neighbours. 

Section 2 provides a brief description of the work that was undertaken as part of 
the consultation exercise and summarises the key findings. Section 3 discusses 
the technical issues raised by the consultation and presents a summary of the 
technical approach used to develop the statistical neighbour model. Section 4 
explores the similarities and differences between the new model and the model 
developed by Ofsted in 1998. Further details on the consultation and technical 
exercises, along with details of the statistical neighbours designated to each LA 
and information about the Excel tool developed for presenting statistical neigh- 
bours can be found in the appendices. 
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2 Consultation 



The first, and crucial, stage of the present research involved gathering the views 
of a range of key stakeholders, including the end-users of statistical neighbours, 
to inform the development of the new model. This section of the report outlines 
the consultation activities and emerging findings. There were three main activ- 
ities that formed part of the consultation with stakeholders: 

• focused discussions with stakeholders via an online ‘bulletin board’ 

• one-to-one interviews with representatives from key national agencies 

• piloting the draft models with LAs. 



2.1 About the bulletin board 

The online bulletin board approach to consultation ensured the burden on indi- 
viduals, LAs and other key agencies was kept to a minimum, but provided 
regular opportunities for stakeholders to comment upon the key issues and share 
their opinions on potential statistical models (anonymously) throughout the 
development stage in a time-efficient manner. The bulletin board was hosted on 
the NFER’s website from May to July 2006 and was accessed via a secure login 
and password by stakeholders who chose to participate. 

The aim was to recruit a group of approximately 150 participants who would 
represent both those likely to be required to provide and update the data used to 
define statistical neighbours, and those who make use of statistical neighbours 
in their work. A total of 204 individuals signed-up to participate in the bulletin 
board discussions. Each member was allocated a unique user identification and 
password and received instructions for logging on to the bulletin board via 
email. Although 204 individuals signed-up to participate in the consultation, 
not all posted a response on the bulletin board. Encouraging participants to post 
their comments and views was the main difficulty with the consultation. By the 
end of July 2006, 63 individuals had posted at least one response (but usually 
more) on the bulletin board. Additionally, the research team also received and 
responded to a number of queries and collected some comments through email 
and telephone contact with individuals. Telephone calls to those who signed up 
to the bulletin board but had not posted a response, suggested that the main rea- 
son individuals were not posting responses was because they had limited time 
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due to other work pressures. See Appendix 1 for further information regarding 
the bulletin board consultation process. 

The NFER research team initially posted three questions, shortly followed by a 
fourth, for discussion and consultation on the bulletin board. These were: 

• [Ql] : There are a number of statistical neighbour benchmarking models in 
use (such as those designed by CSCI and Ofsted). What do you think of 
them? 

• [Q2] : The purpose of this study is to create a single common statistical model 
that’s applicable across children’s services. How realistic is this? 

• [Q3] : What types of information do you think are most important for defining 
statistical neighbours? 

• [Q4]: How often should the information, used to create statistical neighbours, 
be updated in order for the model to remain relevant? 

A further set of questions and an example of the NFER’s statistical tool were 
later added to the bulletin board. This strategy allowed the research team to take 
a developmental approach to the research. As data were collected, it was possi- 
ble to ask more targeted questions. Subsequent questions included: 



Relevance for children’s services 

• [Q5] : Some people suggested it would be useful to have a model which cre- 
ates statistical neighbours separately for each of the five Every Child Matters 
outcomes. Would you prefer a model that creates statistical neighbours for 
each of the five outcomes separately, or a model that creates overall statisti- 
cal neighbours, which bring together the five outcomes? 



Updating the information 

• [Q6]: We asked for comments on how often information used to create 
statistical neighbours should be updated, in order for the new model to 
remain relevant. A five-year cycle is preferred by those who have left com- 
ments so far, as this ensures some stability in groups of neighbours and 
allows for year-on-year comparisons to be made. However, a few respon- 
dents suggested that the model should be updated more regularly to ensure 
it reflects up-to-date data. Which do you consider to be more important: 
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stability (i.e. through updates every five years) or up-to-date data (i.e. 
through updates annually)? 

Types of information 

• [Q7]: We asked for comments on the types of information that are most 
important for defining statistical neighbours. The comments so far suggest 
that deprivation, ethnic diversity, population density and population size are 
the most important for defining statistical neighbours. Do you agree with 
this, or are there other measures that you think are more important? 

• [Q8] : We have received responses that suggest that some demographic indi- 
cators are inappropriate for rural areas. Do you agree with this? And if so, 
which indicators do you feel are inappropriate? 



Example tool 

• [Q9]: The tool is based on a number of demographic variables. Are any of the 
variables included inappropriate for defining statistical neighbours? 

• [Q 10] : Do you think that there is any information not included here that is 
crucial? 

• [Q 11]: The current set of weights were selected by giving greater weight to 
variables that are more strongly correlated with the outcome variables. Do 
you have any views on the amount of weight different variables should be 
given? 

2.2 About the interviews with key stakeholders 

Ten interviews with key stakeholders were also conducted during May and June 
2006. These included representatives of national organisations (government and 
non-government) relating to services for children and young people, as well as 
two Directors of Children’s Services (from one large shire county and one uni- 
tary authority). Three of the interviewees were from organisations that produce 
statistical neighbour models. The interviews explored the ways in which statis- 
tical neighbours are currently used, and how interviewees would like them to be 
used in future, as well as what they viewed the most important features of any 
system of statistical neighbours to be. 
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2.3 About the piloting 



Following the bulletin board consultation and stakeholder interviews, a sum- 
mary of the key findings was made available to bulletin board users and all 
users were invited to comment on the findings to date. A few individuals chose 
to do so: some made specific suggestions, but generally, the feedback suggested 
that the developments were ‘heading in the right direction’. 

Piloting of the draft models took place in September 2006. Four LAs were 
invited to pilot the two potential versions of the tool, and the accompanying user 
guidelines. The LAs were selected to represent the different types of authorities 
(unitary, metropolitan, shire county and London borough) and were spread 
across the country. Representatives from all four LAs had already participated 
in the earlier consultations and were therefore familiar with the purpose of the 
work. Three of the four LAs agreed to participate in the piloting and each pro- 
vided written feedback, which informed the final version of the model. 

2.4 Summary of key findings from the consultation 

See Appendix 1 for further information regarding the findings from the bulletin 
board consultation and Appendix 2 for detailed findings from the interviews 
with key stakeholders. Below is a summary of the key findings from both 
aspects of the consultation. 

• Stakeholders recognised the need for a new single model that is relevant across 
children’s services, but as there are currently a number of existing models in use, 
the relevance and capabilities of a new model needs to be widely promoted to 
potential users. Concerns were raised regarding the challenge of creating a 
model that would satisfy all parties and cover all five ECM outcomes. 

• There was a general consensus that stakeholders would prefer a model that 
brings together the five ECM outcomes, rather than a model that creates sta- 
tistical neighbours for each of the five outcomes separately. 

• Stakeholders made both positive and negative comments about the existing 
Institute of Public Finance (CSCI) and Ofsted models. Generally, they were 
valued for supporting performance improvement work. 

• Aspects of the Ofsted model that were highlighted as positive were that it 
defines the relative degrees of closeness (e.g. very close, close) and has pro- 
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vided consistent comparisons for statistical neighbours. The fact that Euclid- 
ean distances were used was something the model was both appreciated for 
and criticised for. 

• The Ofsted model was criticised for being out of date, relying on data that 
was considered no longer relevant. The methodology was felt to be difficult 
to understand and the model was considered to lack reciprocity. 

• Stakeholders suggested that a new model should: 

- incorporate smaller geographical areas within the model 

- ensure there is reciprocity between statistical neighbours 

- have a set maximum distance for statistical neighbours to still be consid- 
ered ‘neighbours’ 

- include degrees of closeness (similar to the Ofsted model) 

- give LAs the opportunity to compare their authority with all other LAs, 
rather than just their statistical neighbours. 

• Stakeholders felt it was important for users to have confidence in the model, 
given there are a number of models currently in use. It was suggested that the 
model should produce statistical neighbours which are those LAs ‘see’ as 
similar. There was a general view that the process of developing a new model 
needs to be transparent and the model needs to be accessible and understand- 
able. 

• Stakeholders considered socio-demographic data, socio-economic data and 
health data to be important indicators for defining statistical neighbours. 
There were also suggestions to include a broad range of other types of data, 
including crime data, number of people with English as an additional lan- 
guage (EAL), economic activity, number of refugees, and education-related 
data including student mobility, special educational needs (SEN) and free 
school meals (FSM). Geographical closeness of neighbours was not gener- 
ally considered to be of importance. 

• In terms of the frequency with which information should be updated, the 
stakeholders raised issues regarding the stability of the model, if it were 
updated too frequently, but also regarding the accuracy of the model, if it 
were updated too infrequently. The responses focused on where to find the 
balance between accuracy and stability. Overall, maintaining stability (i.e. 
through updates every five years) was considered to be more important than 
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ensuring the data was updated annually. But, it was suggested that an indica- 
tion of potential changes that could affect the model would be useful 
mid-cycle. 

• Overall, stakeholders thought it was important to both compare a LAs per- 
formance over time and compare its performance against similar LAs, 
depending on what was being measured. Stakeholders suggested that the 
model should make use of existing data that LAs use already. 

• Interviewees commented on using either a single-year or three- to five-year 
averages within the model. The latter was thought to be particularly benefi- 
cial if LAs had spent a year establishing a new system or service, but the 
main point made was that the decision should depend on the type of indica- 
tors being used. 

2.5 Implications for the future model 

The data collected through the consultation, both via the online bulletin board 

and the one-to-one interviews, suggested the following should be considered 

when developing the new statistical model and tool: 

• Time should be taken to communicate the rationale for the model to ensure 
‘buy in’. Users must have confidence in it. This is particularly true with 
respect to explaining how the model will work across all five ECM outcomes. 

• The statistical model should produce overall statistical neighbours, which 
bring together the five ECM outcomes, rather than producing separate neigh- 
bours for each outcome. 

• The model may need to evolve over time. 

• Socio-economic and socio-demographic data is important for defining statis- 
tical neighbours and existing data sources should be utilised in this respect. A 
number of variables should be used to determine deprivation, rather than a sin- 
gle indicator. The model should also use existing data that LAs currently use. 

• LAs would find it helpful to be able to compare districts (or small geograph- 
ical areas) with similar districts in other LAs. 

• Reciprocity between statistical neighbours needs to be considered. 

• There should be a set maximum distance for defining a neighbour, and degrees 
of closeness should be defined to show how statistically close LAs are. 
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• The choice of weightings for different variables needs to be considered care- 
fully and explained clearly to users. 

• The model should be updated every five years, to ensure some stability and 
allow for year-on-year comparisons, but also to ensure it remains relatively 
accurate. 
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3 Technical discussion 



3.1 Issues arising from the consultation 

The national consultation with stakeholders led to a range of issues being raised. 
Many were not directly related to the techniques used to define statistical neigh- 
bours, for example, a number of responses related to more general details of 
performance assessment. However, some of the emerging findings from the 
consultation have immediate relevance for the way in which the statistical 
neighbour model is constructed. This section discusses some of the issues 
raised. 

The consultation findings suggested that a single statistical model should be 
produced which brings together the five outcomes, rather than separate models 
for each of the ECM outcomes. But it is important to be able to demonstrate that 
the single model works well across a range of different outcomes. This issue is 
explored in section 3.3. 

A number of LAs suggested that they would find it helpful to be able to compare 
districts (or small geographical areas) with similar districts in other LAs. 
Although, this clearly would be a valuable resource, the project team felt that 
the analysis involved in constructing such a tool would be somewhat different to 
the work required to produce statistical neighbours for LAs as a whole. This 
idea is recommended for further work. 

The findings from the consultation also suggested that there should be a set 
maximum distance for defining a neighbour, and degrees of closeness should be 
defined to show how statistically close LAs are. After further consideration, the 
idea of setting a maximum distance for defining a neighbour was rejected. Work 
looking at the ideal number of statistical neighbours to assign to each authority 
revealed the danger of assigning small numbers of statistical neighbours. 
Benchmarks based on less than five neighbours can be misleading as these place 
too much emphasis on the performance of too few neighbouring authorities. 
But conversely, the degree of closeness of each statistical neighbour has been 
calculated and incorporated into the model to enable LAs to understand the 
nature of their statistical neighbours in greater detail. 
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The consultation revealed a general concern that the process of developing the 
new model needed to be transparent and the model needed to be accessible and 
understandable. In order to make the process of choosing and weighting vari- 
ables transparent, a largely empirical process has been followed. This is fully 
detailed in section 3.2 and appendix 5. Findings from the consultation and liter- 
ature review informed the variables that were initially considered to be used to 
define statistical neighbours. 

The literature review suggested that variables used to define statistical neigh- 
bours should only include those variables which are clearly beyond the control 
of children’s services at the local level. As a result certain variables such as the 
percentage of children with special educational needs were removed before the 
empirical work was started. 

It was recommended during the consultation process that socio-economic and 
socio-demographic data is important for defining statistical neighbours and a 
number of variables should be used to determine deprivation, rather than a sin- 
gle indicator. As suggested, a range of variables covering both material and 
service deprivation were included in the empirical analysis. In order to maintain 
simplicity, each variable was included as a stand alone measure of deprivation 
rather than being combined to form new indices of deprivation. Although some 
use was made of the ODPMs Indices of Multiple Deprivation, many of these 
indices were inappropriate for use since they made use of ECM outcome data in 
their definitions. For example, since the Education, Skills and Training depriva- 
tion domain score uses results of national tests in its definition, defining 
statistical neighbours using this index would lead to circular reasoning. 

The literature review raised the issue that the effectiveness of various depriva- 
tion indices could be different within rural settings. For this reason, a number of 
measures relating to the rurality of LAs were included in the empirical analysis. 
Including these variables in the statistical neighbour model increases the 
propensity for LAs with large rural populations to be compared to LAs in simi- 
lar circumstances and so should effectively deal with this issue. 

During the consultation process a number of stakeholders commented that is 
was desirable that every statistical neighbour they were compared to, was also 
compared to them (resulting in reciprocity between statistical neighbours). This 
would involve splitting LAs into groups, rather than having a separate set of sta- 
tistical neighbours for each LA. Each LA would then be the statistical neighbour 
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of every other local authority within their group. It was proposed that defining 
statistical neighbours this way would help to improve information sharing 
between neighbours. In order to explore this possibility, a second statistical 
neighbour model was developed that split LAs into groups. This model was 
then compared to the more conventional statistical neighbour model, to explore 
the impact of this approach on the amount of difference between LAs and the 
statistical neighbours they were compared to. A decision on whether to ensure 
‘reciprocity’ between statistical neighbours was then made. This exercise is 
described in section 3.4. 

3.2 Defining statistical neighbours 

The process of defining statistical neighbours is described in detail in appendix 
5. The main part of the process involved deciding what information should be 
used to define statistical neighbours and how much weight should be given to 
different pieces of information in this process. A number of sources of informa- 
tion were explored through a data mapping exercise. This process was informed 
by the consultation with local authorities and other stakeholders, the literature 
review and by examining which variables were used in existing statistical 
neighbour models. 

Once data from all the suggested sources had been collated, a four-stage process 
was used to produce the final statistical neighbour model. 

1. Regression analysis of performance indicators on the proposed background 
variables. This process was used to reveal which performance indicators were 
appropriate for benchmarking using a statistical neighbour model and which 
background variables had little or no association with performance. Variables 
found to have a weak association with outcomes were discarded. 

2. Analysis of correlations between background variables. Where high correla- 
tions existed between two background variables, one of these was removed. 
The variable kept in the model was chosen on the basis of ease of calculation, 
regularity of updating and strength of relationship with outcomes revealed in 
stage 1. 

3. Calculating the appropriate number of statistical neighbours to assign to each 
local authority. Analysis explored the robustness of models which assigned dif- 
fering numbers of statistical neighbours to each LA. It was decided that 
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assigning ten statistical neighbours to each LA provided a robust model whilst 
maintaining simplicity. 

Assigning weights to each of the remaining background variables in order to 
define statistical neighbours. Weights were assigned to background variables so 
as to minimise the expected difference between each LA and its statistical 
neighbours across a range of outcome indicators. 

The variables included in the final statistical neighbour model and the weight 
assigned to each variable is shown in appendix 5. 



Further validation 

It is clearly desirable that the newly derived model provides a sensible bench- 
mark across all potential outcome indicators. This issue was explored by 
calculating the average squared difference between the performance of each LA 
on each outcome indicator and the average performance of its statistical neigh- 
bours. Results for the new model were compared to the results for the IPF 
comparator councils, Ofsted’s statistical neighbours and a model based on 
assigning equal weights to each background variable. This analysis revealed 
that for all outcomes, with the exception of two, the new proposed model pro- 
vided benchmarks for local authorities that were at least as accurate as any other 
statistical neighbour model. In the two cases where the model was less accurate 
than others, the differences were very slight. 

From this evidence it can be assumed that the statistical neighbour model 
derived through this process will provide a reasonable benchmark for perform- 
ance across most ECM outcomes. Further details on this analysis can be found 
in appendix 5. 

Using local authority ‘families’ rather than a 
statistical neighbour model 

As mentioned in section 3.1, a second statistical neighbour model was created 
that split LAs into families of authorities and thus ensured reciprocity between 
statistical neighbours. This model was based on the same variables and variable 
weights as the statistical neighbour model already described in this section. 
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The two models were compared to explore the extent to which the average dis- 
tance between statistical neighbours and the overall expected difference in the 
performance of statistical neighbours increased, when families were used in 
place of conventional statistical neighbours. The analysis revealed that the aver- 
age Euclidean distance between statistical neighbours increased by 44 per cent 
and the expected difference in outcome measures increased by around 12 per 
cent. This result is perhaps to be expected as many LAs will be near the bound- 
aries of families. Subsequent to this analysis, the use of LA families for defining 
statistical neighbours was rejected. 
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4 Comparison of the new model 
with Ofsted’s model 

At present the most widely used statistical neighbour model is the one devel- 
oped by Ofsted in 1998. This section explores the ways in which the new model 
developed by NFER differs from this. 

It is worth noting that the Ofsted model was devised with the aim of providing 
LAs with comparative information for purely educational outcomes. This is in 
contrast to the new model which is intended to provide comparisons across a 
range of indicators relevant to the ECM agenda. This change in the intended 
puipose of the model results in some important differences in the ways in which 
statistical neighbours are defined. 

4.1 Variables used 

In total seven of the variables used to define statistical neighbours in the new 
model are in common with variables used in the Ofsted model, six variables 
from the Ofsted model have been replaced with alternative but similar measures 
and five have been discarded. Table 4.1 gives more detail on which variables 
have been kept, discarded and replaced. It should be noted that the Ofsted model 
is based on data from the 1991 census whereas the new model uses 2001 census 
data. As a result, even where identical measures are used the actual data 
included in the model is likely to be different. For more details on the methods 
used for variable selection see section 3 and appendix 5. 

In addition to the changes noted above, six entirely new variables have been 
included in the model. Each of these variables is strongly related to outcome 
measures and not greatly correlated with any of the other variables in the model. 
These variables are: 

• % of dependent children in households where household reference person is 
in any professional or managerial occupation 

• mean gross weekly pay 

• % of vehicles that are three years old or less 

• % of dependent children in one adult households 
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% of people in good health 

% of households owned outright or owned with mortgage. 



Table 4.1 Variables included in the Ofsted model 



Variable used in Ofsted model 



Kept/Replaced/Discarded 



% of households where HOH is in Registrar 
Generals Group IV or V 

% of households with dependent children and 
no car 

% of pupils in maintained schools eligible for free 
school meals 

% of households in dwellings with 7 or more 
rooms 

% of households with 3 or more children 

% of households with more than 1 .5 persons per 
room 

% of households with dependent children, moved 
in the previous 12 months 

% of adults with higher educational qualifications 

Ethnicity information (% of people of white, black, 
Indian, Pakistani, Bangladeshi, other Asian 
ethnicity) 

Population density 

Population growth/decline: % change 

% of population in rural areas or urban 
settlements of less than 20,000 

Number of pupils in state maintained schools 



Replaced with % dependent children in households 
where household reference person is in any routine 
occupation. 

Replaced with % dependent children in households 
with 2 or more cars. 

Variable kept in new model. 

Replaced with % dependent children living in 
households with occupancy rating of +2 or more. 

Variable kept in new model. 

Replaced with % dependent children living in 
overcrowded households. 

Discarded from model due to weak relationship 
with outcomes. 

Variable kept in new model. 

Black ethnicity split into Caribbean, African and 
other. Mixed ethnicity included in new model. 

White ethnicity dropped as a separate category since 
information on other ethnic groups defines this. 

Discarded from model due to weak relationship 
with outcomes. 

Discarded from model due to weak relationship 
with outcomes. 

Replaced with % of the population living in 
villages, hamlets or isolated settlements. 

Discarded from model due to weak relationship 
with outcomes. 



4.2 Changes for individual authorities 

Given the changes noted above, it is to expected that individual authorities’ sta- 
tistical neighbours will change somewhat. Having said this, it would be 
alarming if there was no relationship between the statistical neighbours pro- 
duced by Ofsted and those defined by the new model. 



the development of the Children’s Services Statistical Neighbour Benchmarking Model 



17 



In order to investigate this issue, the team counted the number of statistical 
neighbours that were assigned by both the Ofsted model and the new model, for 
each LA. Of the 1500 statistical neighbours that are assigned in total by the 
new model (10 for each LA) 683 (46%) had also been assigned to the same LAs 
by the Ofsted model. Further details on this can be seen in table 4.2. 



Table 4.2 Degree of change from Ofsted model for individual LAs 



Number of similar neighbours 


Number of LAs with this number of neighbours 
in common between the Ofsted model and the new model 


0 


1 


1 


3 


2 


9 


3 


28 


4 


31 


5 


34 


6 


29 


7 


10 


8 


5 


Total 


150 



Generally there is reasonable agreement between the models with over half of 
LAs having five or more statistical neighbours in common in both models. Only 
one LA (Medway) has been assigned a completely changed set of statistical 
neighbours. This apparent disparity could be caused by the fact that there are 33 
local authorities defined as being ‘Very Close’ (see appendix 8) to Medway and 
it is difficult for any model to pick between these. 
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Appendix 1 The bulletin board 
consultation 



A1 .1 About the bulletin board 

For the statistical model to be useful to the end-users, a crucial part of this 
research involved gathering the views of a range of key stakeholders, including 
the end-users. One way in which this was achieved was through an online dis- 
cussion forum or ‘bulletin board’. This approach to consultation ensured the 
burden on individuals, LAs and other key agencies was kept to a minimum, but 
provided regular opportunities for stakeholders to comment upon the key issues 
and share their opinions on potential statistical models throughout the develop- 
ment stage in a time-efficient manner. The bulletin board was hosted on the 
NFER’s website and accessed via a secure login and password by selected 
stakeholders. This enabled stakeholders to participate in the consultation at any 
time, as convenient. The bulletin board was monitored by the research team on 
a regular basis. 



A1.2 Recruitment 

The bulletin board went live on 5 May 2006. A total of 495 individuals working 
in organisations related to children and young people’s services were emailed 
and invited to take part. This included individuals from the following organisa- 
tions: Community Foundations, Crime Reduction Partnerships, Drug Action 
Teams, Learning and Skills Council, Neighbourhood Renewal Teams, Regional 
Government Offices, Strategic Health Authorities, Sure Start and Youth Offend- 
ing Teams. These individuals were asked to open the invitation to their 
colleagues. 

A message was posted on EMIE at NFER’s eddie discussion forum for LAs, 
inviting bulletin board participation 1 . The discussion forum reaches 480 individ- 
uals. The research team also made use of contacts in various networks. Eighty 
people within the Local Education Authorities Research Group (LEARG) and 
London Education Research Network (LERN) were emailed and invited to take 
part. An email was also sent to 150 EMIE link officers (one per LA) 2 . In addi- 
tion, flyers advertising the consultation were distributed to delegates who 
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attended children’s services data workshops at the NFER (approximately 80 
individuals). Lastly, an advert was posted directly on the EMIE website. 3 

On 19 May (two weeks after the bulletin board went live), the research team 
posted a new question on the bulletin board. An email was sent to all individu- 
als who had signed-up to inform them of this change and to remind them to post 
their comments. A further week later, on 26 May, a short email reminding indi- 
viduals to post their comments was sent, emphasising that the initial 
consultation must end on 14 June. On 2 June, all Directors of Children’s Serv- 
ices (who had not already been directly contacted via other means) were 
emailed and invited to participate in the bulletin board. They were also given 
opportunity to feed their comments directly to the research team or to pass the 
invitation on to their colleagues. On the same day, an advert was placed in Con- 
fed’s (The Confederation of Education and Children’s Services Managers) 
electronic newsletter to members, inviting participation in the consultation. 

On 16 June a new set of questions were added to the bulletin board for further 
discussion. This also included an example model for stakeholders to download 
trial and comment upon. Table A 1.1 summaries these activities. 

A1 .3 Response rate 

The aim was to recruit a group of approximately 150 participants who would 
represent both those likely to be required to provide and update the data used to 
define statistical neighbours, and those who make use of statistical neighbours 
in their work. A total of 204 individuals signed-up to take part in the bulletin 
board consultation. Each member was allocated a unique user identification and 
password and received instructions for logging on to the bulletin board. 
Although over 200 individuals signed-up to participate in the consultation, not 
all posted a response on the bulletin board. Encouraging participants to post 
their comments and views was the main difficulty. 

As of 30 June, 58 individuals had posted at least one response on the bulletin 
board. The research team also responded to a number of queries and collected 
some comments through email and telephone contact with individuals. This was 
additional to the bulletin board responses. In order to increase the response rate, 
telephone calls were made to 101 individuals who signed up to the board, but 
had not posted a response. This acted as a gentle reminder, and has also helped 
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to determine the reasons for the non-response. The information gleaned from 
these telephone calls suggested that the main reasons individuals were not post- 
ing responses were time constraints. Table A1.2, in the subsequent section, 
shows the number of users who responded, by question. 



Table A1 .1 Recruitment activity 



Week 


Date 


Activities 


1 


5 May 


• Bulletin board went live 

• 495 individuals from a range of organisations were emailed and invited to take 
part 

• A message was posted on the eddie discussion forum inviting participation 

• 80 people within LEARG and LERN were invited to take part 

• 150 link officers were emailed 

• Flyers were distributed to 80 delegates at children’s services data workshops 

• Individuals who wished to take part were sent login details (continuous) 


3 


19 May 


• A new question was added to the bulletin board for discussion 

• All individuals who had signed-up to the bulletin board were emailed again 
and informed of this 


4 


26 May 


• A short email reminder was sent to all individuals who had signed-up 


5 


2 June 


• All directors of children’s services (who had not already been contacted) were 
emailed and invited to take part 


6 


9 June 


• Telephone calls were made to individuals who had signed-up but not posted a 
response 


7 


16 June 


• A new set of questions, including an example tool for download, were added 
to the bulletin board and all individuals who had signed-up were emailed 
about this 


9 


30 June 


• All bulletin board responses were transferred for qualitative analysis 



A1.4 Questions asked 

The NFER research team initially posted three questions, shortly followed by a 

forth, for discussion and consultation on the bulletin board. These were: 

• [Q 1] : There are a number of statistical neighbour benchmarking models in 
use (such as those designed by CSCI and Ofsted). What do you think of them? 

• [Q2] : The purpose of this study is to create a single common statistical model 
that’s applicable across Children’s Services. How realistic is this? 

• [Q3] : What types of information do you think are most important for defining 
statistical neighbours? 
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• [Q4] : How often should the information, used to create statistical neighbours, 
be updated in order for the model to remain relevant? 

A further set of questions and an example of the statistical tool were added to 
the board on 16 June. Again, stakeholders were informed of these changes by 
email. This strategy has allowed the research team to take a developmental 
approach to the research. As data has been collected, it has been possible to ask 
more targeted questions. Subsequent questions included: 



Relevance for children’s services 

• [Q5] : Some people suggested it would be useful to have a model which cre- 
ates statistical neighbours separately for each of the five Every Child Matters 
outcomes. Would you prefer a model that creates statistical neighbours for 
each of the five outcomes separately, or a model that creates overall statisti- 
cal neighbours, which bring together the five outcomes? 



Updating the information 

• [Q6] : We asked for comments on how often information used to create statis- 
tical neighbours should be updated, in order for the new model to remain 
relevant. A five-year cycle is preferred by those who have left comments so 
far, as this ensures some stability in groups of neighbours and allows for 
year-on-year comparisons to be made. However, a few respondents sug- 
gested that the model should be updated more regularly to ensure it reflects 
up-to-date data. Which do you consider to be more important: stability (i.e. 
through updates every five years) or up-to-date data (i.e. through updates 
annually)? 



Types of information 

• [Q7] : We asked for comments on the types of information that are most 
important for defining statistical neighbours. The comments so far suggest 
that deprivation, ethnic diversity, population density and population size are 
the most important for defining statistical neighbours. Do you agree with 
this, or are there other measures that you think are more important? 
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[Q8] : We have received responses that suggest that some demographic indi- 
cators are inappropriate for rural areas. Do you agree with this? And if so, 
which indicators do you feel are inappropriate? 



Example tool 

• [Q9] : The tool is based on a number of demographic variables. Are any of the 
variables included inappropriate for defining statistical neighbours? 

• [Q 10] : Do you think that there is any information not included here that is 
crucial? 

• [Q 11]: The current set of weights were selected by giving greater weight to 
variables that are more strongly correlated with the outcome variables. Do 
you have any views on the amount of weight different variables should be 
given? 



Table A1 .2 Number of respondents by question 



Question number 


Number of respondents 


1 


12 


2 


13 


3 


9 


4 


12 


5 


11 


6 


10 


7 


6 


8 


1 


9 


5 


10 


1 


11 


2 



A1 .5 Analysis 

The data collected through the bulletin board was copied and transferred from 
the bulletin board for qualitative analysis. The responses are summarised below 
in sections A 1.6 through to A 1.10. 
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A1.6 Views on existing models 



One of the initial questions that was posted on the bulletin board was designed 
to prompt discussion about existing statistical neighbour models, in order to 
understand both the ways in which stakeholders found the models useful, and 
where they had concerns, to inform the development of the new model. Stake- 
holders who chose to respond to this question tended to be individuals based in 
local authorities (from both education and social care backgrounds), and were 
predominantly responsible for either performance management or information 
and statistics. 

• The respondents made both positive and negative comments about the CSCI 
and Ofsted models and they also made suggestions about specific criteria that 
they felt needed to be taken into consideration when designing a new model. 

• Whilst commenting on existing models, it is worth noting that a number of 
stakeholders chose to make comments about the need for a new single statis- 
tical neighbour model relevant across children’s services, illustrating support 
for the overall aims of this project. Some of the comments were related to it 
being confusing having several models with different sets of neighbours 
within each; others merely stated that with the advent of children’s services, 
it was preferable to have a single model that is relevant across children’s 
services. For example, one respondent wrote ‘given the range around, we 
really need one model used by all agencies involved with children’s services’ 
and another noted ‘we need just one agreed set of neighbours’. 

• A few respondents were questioning the purpose of statistical benchmarking 
more generally and felt that it was the use that was made of statistical neigh- 
bour comparisons that was problematic rather than the models themselves, 
particularly when the comparisons were used by those outside of the LA to 
make judgements. This suggests that when disseminating the new model, the 
explanations and caveats that accompany it will be of utmost importance, if 
the model is to be widely accepted and used across authorities. 

• Existing models were appreciated by stakeholders for supporting perform- 
ance improvement work generally and specifically for highlighting areas 
where a LA was performing well and areas where improvements were 
needed. One stakeholder remarked ‘whilst the statistical neighbour data can- 
not provide the only basis for comparison, they are an important tool in 
benchmarking ourselves against the performance of “LAs like us”‘. 
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• Aspects of the Ofsted model which were particularly highlighted as positive 
were that it: 

- defines the relative degrees of closeness (e.g. very close, close) 

- has provided consistent comparisons for statistical neighbours 

- uses Euclidean distances 4 . 

• Aspects of the Ofsted model which were felt to be problematic were that: 

- it was considered out of date (e.g. using 1991 Census data) and therefore 
‘inflexible’ and it does not match other national datasets 

- it relies on data considered to be of limited value or no longer relevant 

- the methodology was considered difficult to understand 

- lacks reciprocity, as each LA has a unique grouping so LA X may have LAY 
as a statistical neighbour, but LA Y may not have LA X as their neighbour 5 . 

• A general criticism of statistical neighbour models made by respondents was 
that they struggled to balance the different input variables appropriately. 

• Whilst commenting upon the existing statistical neighbour models, many 
stakeholders made suggestions on specific criteria they would like included 
in a new model. The main suggestions included: 

- incorporating smaller geographical areas within the model, such as dis- 
tricts within a LA, to enable LAs to ‘drill down’ and identify variation 
within the authority. Stakeholders suggested that it would also be useful to 
be able to compare these smaller areas/districts with those in other LAs. As 
one respondent explained: 

Our LA is demo graphically very polarised, containing some very deprived 
areas and also some of the most affluent in the region. We could better 
evaluate our performance if we could compare these areas separately with 
similar areas in other LAs. 

- ensuring some degree of reciprocity within the comparisons. One respon- 
dent explained why this was important: 

I turn down most requests from other LAs for joint benchmarking work as 
although my authority is included in their group, the other authority is not 
included in mine. This needs to be sorted out if authorities are to use 
benchmarking for qualitative work rather than just numerical comparison. 
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- having a set maximum distance (using the Euclidean measure), to prevent 
LAs being compared with statistical neighbours that are too different from 
themselves, thereby reducing the number of statistical neighbours in some 
cases 

- incorporating degrees of closeness into the model (e.g. close, extremely 
close) so it is clear how similar other LAs are 

- giving LAs the opportunity to compare their authority with all other 
authorities, rather than just their statistical neighbours. 

• Other responses included discussion about having confidence in the model. It 
was felt important that the users have confidence in the model, in that their 
statistical neighbours are those that they ‘see’ as similar. To address this 
would require piloting the model with a sample of LAs. 

A1 .7 Views on information types 

Stakeholders were asked about what types of information they thought were 
most important for defining statistical neighbours. Generally, respondents 
thought that socio-economic data like deprivation, ethnic diversity, population 
density and population size were the most important types of information for 
defining statistical neighbours. However, respondents to the bulletin board 
entered an extensive debate about issues such as which indicators could be used 
to provide such measures and the need to make appropriate use of existing 
sources, such as that available through ACORN, Ofsted, ODPM, CSCI and 
other sources. This was summarised by one stakeholder’s plea for the new 
model to avoid the need to ‘reinvent the wheel’. 

Respondents also debated the level and sophistication of the measures needed - 
which linked to previous responses in that overall the implication was a need to 
consensually decide the focus for the model to identify appropriate measures. 

To further explore the types of data stakeholders wanted the new model to con- 
tain, they were asked a supplementary question about whether there were 
‘other’ types of data (other than socio economic measures) that would be useful. 
Among the suggestions left on the bulletin board were: 

• A suggestion that a more extensive list would enable LAs to look at the 
specifics for individual departments or services, e.g. education. 
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• A broad agreement that more educationally orientated data would be useful 
such as special educational needs (SEN) and free school meals (FSM) infor- 
mation. 

• Suggestions for a broad range of types of data covering health, crime, number 
of people with English as an additional language (EAL), student mobility, 
economic activity and the number of refugees. 

• A suggestion that using demographic data for rural areas might not prove a 
fair and useful measure on which to base comparisons. 

A1.8 Views on updating the information 

Stakeholders were invited to respond to a question about the frequency with 
which information used to create statistical neighbours should be updated in 
order for the new statistical model to remain relevant. 

• The respondents raised issues regarding the stability of the model, if it were 
updated too frequently, but also regarding the accuracy of the model, if it 
were updated too infrequently. The responses focused on where to find the 
balance between accuracy and stability. 

• More than half of the respondents suggested that the model would need 
updating every five years, to ensure some stability. For example, one stake- 
holder noted that ‘robust statistical data is not available promptly enough to 
facilitate regular updating of well-defined statistical neighbours, so a five 
yearly update feels appropriate’. 

• A small number of respondents argued that updates would need to be more 
regularly than every five years, for example, to link in with PLASC updates. 
Or as one stakeholder noted: ‘more frequent data is never bad, but you do 
need to know your data and your area’ . 

• None of the respondents suggested less frequent updates than five-yearly, 
although one respondent was unconvinced that it would be possible to update 
even as frequently as every five years. 

Following on from these findings, a further question was posted on the bulletin 
board asking specifically whether stability (i.e. through updates every five 
years) or up-to-date data (i.e. through updates annually) was more important. 
These further responses confirmed what the initial responses suggested: 
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• Maintaining stability was clearly considered by the stakeholders to be more 
important than ensuring the data was updated annually. One reason for this 
was that ‘constant changes make year on year comparisons very difficult’. 

• Some stakeholders also noted the importance of the model, and its cycle, 
remaining simple. 

• It was suggested that although a five-yearly cycle would be most appropriate, 
an indication of potential changes, perhaps at the mid-point of the cycle 
would also be helpful, e.g. if there were variables that were likely to affect the 
neighbour model that changed significantly during the cycle. 



A1 .9 Views on relevance for children’s services 

Stakeholders were asked if they would prefer a model that creates statistical 
neighbours for each of the five outcomes separately, or a model that created 
overall statistical neighbours, which would bring together the five outcomes. All 
respondents said that they wanted one model; one respondent said including all 
five outcomes would be more accurate, but lead to confusion among the 
‘troops’. 

Stakeholders were also asked whether creating one model was a realistic ambi- 
tion. Generally, respondents agreed that creating a single model was a 
‘necessity’ to facilitate common and informed debate - although several respon- 
dents said that the model was part of the answer rather than an end in its self. 
Respondents entered into an extensive discussion on this issue, such as: 

• Questioning whether the model should provide ten neighbours and not more. 

• Stating that in creating a single model, compromise would be a key element. 

• Highlighting the pitfalls of not having one model, such as the problems posed 
if one LA should be compared with one set for one aspect of its performance 
and another set for another aspect. 

• Raising the challenges posed in creating a single model, summarised by one 
respondent thus: 

. . . the model will be difficult to achieve. It can only be developed over-time 
rather than introducing new Pi’s / key indicators. Perhaps we should carry 
on with some of our tried & tested Pi’s and gradually phase in additional 
ones? Each LA is different and in some circumstances it maybe difficult to 
compare themselves with Statistical neighbours & nationally due to small 
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number of children involved e.g. looked after children; ethnic minority 
groups; & other vulnerable groups. Also issues around the frequency & 
availability ofstat. neigh. & national datasets - LAs need access to this infor- 
mation much earlier? 

• A suggestion that with increasing coterminousity of PCTs with LAs as a 
result of PCT reorganisation, the statistical benchmarking ought to reflect 
this. 

• A general view that the process of development needs to be transparent. 

• The need to update the model to reflect changing needs and priorities. 

• An extensive debate about the level of and number of factors to be included 
and their relevance and the sophistication of the model. Overall, this debate 
highlighted the importance of agreeing a starting point or principles with an 
explicit focus and clarity about what the model is intending to achieve and 
perhaps what it can not achieve. 

A1.10 Views on the example tool 

Stakeholders were provided with a ‘downloadable’ example of a model, and a 

list of potential variables that might be included in a model and asked which 

they thought would be appropriate for defining statistical neighbours. Com- 
ments posted on the bulletin board showed that respondents thought that: 

• The model should attempt to use and retain consistency with existing data 
that LAs use, such as PANDA. 

• To indicate levels of deprivation it would be preferable to use ‘representative’ 
measures that bring together certain variables associated with deprivation, 
rather than single variables such as free school meals. Suggestions for exam- 
ples of a more sophisticated approach included ACORN, IMD and ID ACL 

• A number of additional variables facilitating neighbour comparison/differen- 
tiation were suggested, including local Gross Domestic Product (GDP), 
access to authority services domains, health and crime. 

• The link between ‘input’ variables and ‘outcome’ indicators should be made 
clear. 

• To accompany any new model there should be a clear and transparent discus- 
sion about how decisions about the model had been reached. 
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Following on from these responses, stakeholders were asked if there were any 
variables that were not included in the list of responses discussed above, that 
they would like included. There was only one response, which suggested that 
the number of children living in ‘areas identified as deprived’ would be a useful 
measure to include. 

A summary of the key findings from the bulletin board consultation and the 
implications of the findings can be found in Section 2 of the report. 



Notes 



1 EMIE at NFER is an information service supporting LAs responsible for children’s services. The serv- 
ice is widely used by LA staff, particularly those involved in research and statistics in children’s 
services, eddie is an email discussion forum, managed by EMIE at NFER, for staff working in educa- 
tion and children's services departments within LAs. 

2 EMIE at NFER has a liaison officer in each LA with responsibility for children’s services in England. 

3 There was a small overlap of about 10 people between LERN, LEARG and link officers; about 45 
between eddie and the others. 

4 However, this was also an aspect the model was criticised for. 

5 The CSCI model (Institute of Public Finance model) was also criticised for a lack of reciprocity. 
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Appendix 2 Interviews with 
stakeholders 



A2.1 About the interviews 

Ten one-to-one interviews were conducted with key stakeholders representing 
national organisations (government and non-government) relating to services for 
children and young people, as well as two Directors of Children’s Services (from 
one large shire county and one unitary authority). The interviews explored the 
ways in which statistical neighbours are currently used and how interviewees 
would like them to be used in future, as well as what they perceive to be the most 
important features of any system of statistical neighbours. The interviews were 
conducted during May and June 2006 and the interview notes were transferred to 
MaxQDA, a qualitative analysis tool, to facilitate the analysis of the data. 

A2.2 Roles and responsibilities in relation to data 
collection and use 

In order to gain an overall picture of the nature of involvement that each of the 
stakeholders had in relation to data collection and use of the data, the intervie- 
wees were asked to give a brief description of their roles and responsibilities. 
The interviewees were all involved in quantitative data collection/use to a 
greater or lesser extent. This included: 

• making use of data collected by approximately ten inspectorates, including the 
Home Office, CSCI, Audit Commission, Youth Justice Board and Adult Lean- 
ing Inspectorate. The data was used to focus on areas for further investigation 

• looking at data in order to further develop LA services, working with other 
agencies to establish a common data set by which to monitor performance 

• managing those who were responsible for data collection or analysis 

• validating the data collected by colleagues and managing the organisation of 
the analysis. Although not directly involved in data collection, this intervie- 
wee did make use of data in his role 

• holding discussions with other bodies about data collection processes and 
performance measures, particularly in relation to the data sets used across 
social care, but increasingly health and education too 
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• using data sets to gain information about trends and current issues. This inter- 
viewee regarded his knowledge of statistical neighbour models as ‘very 
limited’ 

• making use of secondary statistical data to inform discussions and policy 
development. 

A2.3 Organisational use of statistical neighbour 
models 

All but one of the interviewees reported that their organisation made use of 
existing statistical neighbour models, either the Ofsted model, the IPF (CSCI) 
model or both. There was, however, variety in the level of usage. Three intervie- 
wees, for example, worked for organisations producing one of the statistical 
neighbour models. One of these interviewees commented that his organisation’s 
model was used in virtually all areas of its work. A further interviewee consid- 
ered statistical neighbours critical to her organisation’s work in order for it to 
monitor its own performance. This organisation also used statistical models for 
the purposes of comparisons, performance management and evaluating value 
for money. 

One interviewee mentioned that his organisation also made use of ODPM 
benchmarking to analyse its own performance. He commented that a bench- 
marking system was useful, but felt using three models was ‘quite confusing’. 
Another interviewee reported that her organisation also made use of the IPF 
model, in order to carry out comparative work. Yet, a further interviewee, a pol- 
icy adviser, explained that statistical data was used in a limited way in his 
organisation. One organisation was not making use of such models as it was 
only recently established. The interviewee from this organisation explained that 
models may be used in the future, if thought useful to specific projects. 

A2.4 Usefulness of current statistical neighbour 
models 

Generally, the interviewees thought the current statistical neighbour models 
were useful, although this was not necessarily in relation to children’s services. 
The interviewee most confident about the usefulness of current models in rela- 
tion to children’s services worked for an organisation which maintained its own 
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model. He felt that, although the model needed updating, it was a good model 
for inspecting certain elements of children’s services. One interviewee simply 
said current models had a ‘role to play’ in judging performance, and another 
remarked they were useful because they provided ‘a pattern of data’. A further 
interviewee was somewhat apathetic, commenting that current models were 
‘very useful because they are the only ones we have got’. She said not having 
any sort of tool would defeat the object of performance management. 

Three interviews pointed out that while it was useful for LAs to compare them- 
selves to others, some caution needed to be applied. They felt local factors 
needed to be taken into account, as ‘apples were not being compared with 
apples’. For example, one interviewee said a LA could cover a large geograph- 
ical area, but serve a small population. Therefore, she felt a new model should 
work on the basis of pupil numbers not percentages. Similarly, she thought a 
new model should consider population distributions. For example, one rural 
authority may have a thinly spread population, whereas another could have 
densely populated pockets. She felt these situations would have implications for 
the delivery of services. Also, the interviewee was of the opinion that a new 
model should consider the amount of funding LAs receive per pupil, because 
this can differ. Another interviewee said ‘we need to identify where the points of 
differences are and where the data diverge’, for example, across age groups. The 
mobility of a population was also mentioned as a factor to be considered. 

With regards to local context, one interviewee commented that the ODPM 
model was particularly useful. He explained that this model is based on smaller 
areas rather than whole LAs, i.e. census wards. The interviewee thought this 
type of information was very important for LAs, especially large shire authori- 
ties, because it takes into consideration local factors, such as urbaness/ruralness. 
Making reference to other models currently available, he said the IPF model 
constrained the statistical analysis of the different types of authorities. For 
example, metropolitan authorities could be compared with unitary authorities, 
but shires could not be compared with London. He said this drawback would 
become irrelevant if a new model used smaller areas instead of whole LAs. 
Overall, this interviewee was of the opinion that ‘disagrability’ was the ‘gold 
standard’ of any statistical model. He suggested this might be possible if unique 
pupil identification numbers were used. 

From the perspective of an interviewee who worked as a policy consultant, he 
felt the data used in current statistical neighbour models might connect to ECM 
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indicators, but he had reservations. He commented that a clear link was needed 
between neighbour indicators and the five ECM outcomes. This interviewee 
remarked that a review was needed in order to determine what data would be 
most useful in informing and/or providing measurements to support improve- 
ment. He felt the data used to create statistical neighbour models needed to be 
‘more rounded’. For instance, data could be used to show how educational 
achievement data interacts with child protection data (e.g. is a child with low 
educational achievement more likely to be an ‘at risk’ from illness, neglect or 
abuse than a child with higher educational achievement). The interviewee was 
also of the opinion that any statistical neighbour model should serve as an inter- 
nal benchmarking tool, as well as a tool for comparisons between LAs. 

A2.5 Possibility of creating a single common 
statistical model 

Overall, interviewees thought it was realistic to create a single common statisti- 
cal model that would be applicable across Children’s Services. One interviewee 
explained that, as the JAR document already collects data on approximately 
290 performance indicators, there would be no problem using a single model. 
She said this information would just have to be categorised into the five ECM 
outcomes. In terms of combining existing models, a second interviewee pre- 
dicted that similar factors would affect education and social services outcomes, 
but this was difficult to determine without looking at the neighbours being pro- 
duced. He felt a single model could only work if education/social service/other 
factors were shown to have similar results. 

There were, however, some reservations. One interviewee remarked that a sin- 
gle model was never going to be perfect. Two commented that a single model 
would never satisfy all parties. One of these interviewees thought ‘each of the 
people who have devised their own [statistical model] will argue their corner 
quite strongly’. The advice given was to consult on the characteristics that 
should define a statistical neighbour model. It was suggested that, while a single 
model would be useful, caveats would be needed about what the model could 
and could not achieve. A further reservation was related to a ‘whole system’ (i.e. 
the multi-agency) attempt to improve LA outcomes. The interviewee was con- 
cerned that there was a danger that outcomes for children would become 
‘divorced’ from the development of a model. To possibly avoid such a pitfall the 
interviewee suggested ensuring that the model could be used to focus on spe- 
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cific ‘groups’ of children, such as looked after children. In this way he also felt 
that statistical data and local/national policy could be taken into account, which 
would make the model more practical and useful in helping achieve and address 
the five outcomes. 

A2.6 Making the model relevant 

The interviewees were of the opinion that consultation (at strategic level and 
service manager level) was the key factor in ensuring that a new model was rel- 
evant to, and accepted by, agencies involved in Children’s Services. One 
interviewee commented that any model would need to have, and build, credibil- 
ity with its users. Therefore, discussion would be needed within and between 
LAs to ensure the model were useful, and that it had practical (e.g. implicit links 
to inspection work) and demonstrable advantages to models currently available. 
In his opinion, this would lead to legitimacy and credibility among users, and 
would ensure the data would be used as intended. He also advised that the ration- 
ale for the collection of data should be carefully considered in terms of how it 
could be used (e.g. within LAs, between groups of LAs or nationally). 

This opinion was echoed by other interviewees. One interviewee said that a 
new model could be ‘imposed’ on users, but this might not result in buy-in or 
ownership. She said thorough consultation would facilitate buy-in, and would 
also help to clarify how the new model was better than existing versions. This 
interviewee also advised that the new model be kept simple. The straightfor- 
ward nature of the new model was also stressed by four other interviewees. One 
of these said it needed to be ‘accessible and understandable’. He also thought 
some consistency, particularly with the ODPM model, would be helpful 
because people were aware of how such a system works. Another remarked that 
some users of statistical models are not familiar with Euclidean statistics. There- 
fore, a new model would need to specify why authority A has authority B as its 
neighbour, but not vice versa. 

In addition, two reservations were put forward. One interviewee thought it would 
be difficult to include issues related to policing because the national picture was 
quite complicated. Another interviewee warned that a new model had the poten- 
tial to be disruptive for LAs. For example, a LA could be placed at the bottom or 
middle of a group when they previously outperformed their neighbours. The LA 
would then need to explain this to its elected members and community. 
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A2.7 Consideration of statistical neighbours in local 
authority inspections 

Generally speaking, interviewees felt LA inspections should give some consid- 
eration to statistical neighbours, but that other evidence should be taken into 
account. There was a feeling that statistical neighbours were useful to determine 
the progress being made by an authority, the questions that should be asked of 
an authority and the areas of future focus. However, there was also a feeling that 
statistical neighbours need to be used in a positive way and assist people in 
moving forward. One interviewee said statistical neighbours ‘should be part of 
the evidence base, but not a definitive because there are always local stories 
which explain things’. Another said attention should be given to statistical 
neighbours, but it should not be a disproportionate focus. Other factors may 
need to be considered, such as a LA’s starting point, national averages, perform- 
ance over time and the quality of LA personnel and leadership. 

One interview gave an example about the need for qualitative data, as well as 
quantitative data. She explained that, if a LA is struggling with a particular serv- 
ice, it would be useful to look historically at LAs which have been in a similar 
position, but have progressed upwards. If only quantitative data were consid- 
ered, it may reveal that improvements have been made, but it would not give 
details about the quality of a LA’s work and its impact. For example, the LA 
may have reduced its number of looked-after children from 15 to ten. In terms 
of statistics, this would show that the LA was moving in the right direction. 
However, it would not reveal what had happened to the five children no longer 
being looked-after. The interviewee remarked, ‘were they just thrown out’, or 
did the LA have a carefully planned strategy for exiting looked-after children 
and moving into independent/semi independent arrangements? A different LA 
might have moved from looking after 15 children to 12, but it may have looked 
more carefully at how these children were moved. Statistically, this LA would 
be seen as achieving less well to the LA which moved from 1 5 children to ten. 

A2.8 Future use of statistical neighbours for 
children’s services 

Thinking about the future of children’s services, interviewees generally said 
they would like statistical neighbours to be used in a similar way to the current 
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situation. That is, statistical neighbours being used for comparisons when LAs 
are looking at performance, a way to identify areas for improvement and a 
resource to help inspectors make better judgements about services. However, 
the new model would need to support the ECM agenda. One interviewee said 
she would like to see the same comparator groups used across all five ECM out- 
comes. She thought there would be ‘tremendous’ logic in this type of approach. 
Another interviewee commented that the new model need not produce one set 
of neighbours per LA, but could produce five relating to the five ECM out- 
comes 1 . She thought this issue should form part of the consultation process 2 . 
Two further interviewees said they would like to see a simpler, more user- 
friendly model. One of these interviewees added that he would like the new 
model to focus on the practical and applied use of data. 

A2.9 Basis for local authority comparisons 

Overall, interviewees thought it was important to both compare a LA’s perform- 
ance over time and compare its performance against similar LAs. For example, 
it might be useful to know that a LA has outperformed others, but if attention is 
paid to the LA in isolation, it might become apparent that it has been allocated 
more government money. Four interviewees mentioned that the decision to look 
at performance over time or performance against similar LAs would depend on 
what they were trying to measure. One interviewee said it was important not to 
be ‘totally obsessed’ with the position of a LA in either case, as this could result 
in losing sight of the LA’s impact on children. 

A2.10 Most important indicators in defining statistical 
neighbours 

Interviewees were asked to report on the indicators they felt were most impor- 
tant in the definition of statistical neighbours. The responses given largely 
reflected interviewees’ personal and/or their organisation’s interest. For exam- 
ple, one interviewee was particularly interested in variables related to education, 
such as free school meals. The range of responses given was also dependent on 
an interviewee’s level of knowledge about the indicators currently used. One 
interviewee thought the indicators used to define statistical neighbours were of 
equal importance because each one formed part of the ‘whole picture’. She did 
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not think it possible to exclude any indicators because each could have a major 
impact on the overall picture. 

The most commonly cited indicators were related to socio-demographics (age; 
ethnicity; single/dual parent families; population density), socio-economics 
(house prices; average wage; nature of local economy; presence of local univer- 
sity, transport links) and health (disability; mental health; drug use). It was 
noted, however, that some of these indicators were more comparable than oth- 
ers. A reason why these indicators were thought to be important was for 
financial comparisons between LAs (such as the reasons why LAs are allocated 
different levels of funding). One interviewee commented that social deprivation 
is known to have impacts on education results and the need for social care. A 
further interviewee mentioned the possibility of double counting some of the 
indicators (mentioned above), and that one may be a proxy for another, but this 
could be ruled out if an appropriate methodology was adopted. 

A2.11 Importance of geographical closeness 

In general, interviewees did not think it was important for statistical neighbours 
to be geographically close. However, one interviewee commented, in creating a 
new model, it should be acknowledged that some people think geographical 
closeness is important. He said LAs may prefer being physically close to their 
neighbour to make visits easier, but such endeavours would be of little value if 
the LAs were not statistically alike. It was also noted that unitary and metropol- 
itan authorities tend to be located next to shire authorities, which are likely to 
have different circumstances 3 . Another interviewee said the decision to include 
geographical closeness as a factor should be based on the aim of the compari- 
son, e.g. a particular theme or intended outcome. It was also noted that members 
of the public may think it useful for statistical neighbours to be geographically 
close. 

A2.12 Strengths and weaknesses of using different 
types of data 

In terms of the strengths of using different types of data within a single common 
statistical model, only one interviewee gave an opinion. He felt the most obvi- 
ous strength was having consistent and common data. Two interviewees 
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commented on the difficulties of using different types of data within a single 
common statistical model. One remarked that it would be difficult to find data 
that were updated annually. A second interviewee thought a single model could 
be strong, as long as variables were only measured once. She said, because serv- 
ices collect data at different times (e.g. based on financial or academic years), a 
new model must ensure the same children are being measured, otherwise data 
would be gained from different cohorts. A further interviewee thought that a 
‘trade off’ was needed between timeliness and precision. Two interviewees 
mentioned that a common language among LA services would be helpful in the 
creation of a single statistical model. 

A2.13 Use of inputs or outputs 

Five interviewees thought statistical neighbours should be based on inputs, but 
three thought outcomes were more important 4 . Of those who favoured inputs, 
one interviewee said this was his preference because inputs were known, but the 
outputs were trying to be determined. Flowever, some caution was given. One 
interviewee said, inputs could easily be counted, but measuring outcomes was 
more difficult. A second interviewee said inputs should be factors that a LA has 
no control over, so they could not be manipulated. He also said some factors 
could be both inputs and outcomes, such as attendance, and suggested exclud- 
ing such factors. A further interviewee remarked that inputs needed to be 
focused on what outcomes were being sought and how these will make a differ- 
ence. 

A2.14 Recalculating data for a statistical neighbour 
model 

The interviewees did not give a fixed timeframe for the recalculation of the 
information used to create statistical neighbours, but it was generally agreed 
that consistency and a degree of certainty was needed. One interviewee said sta- 
bility was needed when using a comparative system and suggested data be 
recalculated every five years, for example following the five-yearly census 
review carried out by the ONS. He said ‘certainty of the outcome’ was an 
important issue when making comparisons. For example, if a LA was inspected 
against certain LAs during one year, and the inspection team returned the fol- 
lowing year looking at a different group of neighbours, their job would be made 
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more difficult. Another interviewee remarked that, while it was tempting to use 
the most current data, this made evaluation over time more difficult because the 
comparator groups would keep changing. In addition, it was noted that regular 
recalculation could be burdensome for LAs. 

A five-year cycle was also mentioned by other interviewees. However, other alter- 
natives to a five-year cycle were offered. One interviewee said a compromise 
would be every three to five years, but the ability to do this was dependent on 
when the information used to create statistical neighbours was updated. A second 
interviewee thought that annual recalculations should be carried out because per- 
formance assessments are now performed once a year. A further interviewee said 
information should be recalculated fairly regularly to take into consideration 
regeneration, and shifts in population (such as immigration patterns). She thought 
ten-yearly data, as in the case of the UK Census, would not be sensitive to interim 
change. It was also noted that data used when creating statistical neighbours 
would always be one year behind. The interviewee suggested this could be partly 
resolved by using a central database where LAs could conduct some of their own 
analysis, and perhaps data awaiting validation could be used. 

A2.15 Using single-year or three- to five-year averages 

Five interviewees commented on the advantages and disadvantages in using 
either single-year or three- to five-year averages. Two interviewees preferred 
three- to five-year averages. This was seen as being particularly helpful if a LA 
had spent a year establishing a new system or service, as the impact could be 
ascertained in the following years. Another interviewee expressed the view that 
if a factor was volatile to change, it ought not to be included in a statistical 
neighbour model. It was also noted that the decision to use either a single-year 
or three- to five-year averages would depend on the indicators being used. 



A2.16 Additional comments 

Interviewees were given the opportunity to provide further comments. These 
included: 

• being clear about the underlying agenda for the new model 

• a new statistical neighbour model needs to be practical and useful for both 
statisticians and policy makers 
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• deciding on the number of statistical neighbours to be allocated to each LA 
(i.e. fixed, not fixed or up to a certain number) 

• testing whether or not there is an overall set of statistical neighbours that are 
good enough to use for all purposes 

• trialling the new model with LAs prior to implementation 

• considering the best strategy for small authorities such as the Isles of Scilly, 
Rutland and City of London 

• examining how benchmarking (internal and external) fits with the creation of 
the new model 

• connecting the present study with existing work to avoid duplication and to 
benefit from other findings. 

A summary of the overall key findings from the stakeholder interviews (and 

bulletin board consultation) as well as the implications of the findings can be 

found in Section 2 of the report. 



Notes 



1 In response to another question, another interviewee warned against this approach. 

2 This issue was addressed as part of the bulletin board consultation. 

3 It was noted in response to a different question that similar LA type did not necessarily imply similar 
circumstances. 

4 During analysis, it became clear than one of these interviewees had taken inputs to mean resources 
rather that demographic variables. 
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Appendix 3 Literature review 



A3.1 Introduction 

To determine which variables should be used to create statistical neighbours, 
the research team reviewed the latest writings in this field to evaluate the 
advantages and disadvantages of using particular indicators. The research team 
is of the opinion that, in practice, not all stakeholders will agree as to which 
indicators should be used, and stakeholders’ expressed preferences may not be 
validated by the statistical modelling. Therefore, the final model or models 
proposed will have to be based on sound evidence obtained from other 
research, as well as from statistical analysis and the consideration of stake- 
holders’ views. 

A3.2 Searching the literature 

In collaboration with the NFER’s Library and Information Service, the 
research team located a range of relevant literature. The databases searched 
included: ASSIA (Applied Social Sciences Index and Abstracts); BEI (British 
Education Index); CERUK (Current Educational Research in the UK); ERIC 
(Education Resources Information Center); PsycINFO and Social Policy and 
Practice, as well as the Library’s own internal bibliographical databases. Inter- 
net searches were also carried out on relevant subject gateways and websites. 
By looking at what has been done in the past, the research team was able to 
consider the usefulness of methods adopted by other organisations, agencies 
and departments. 

A3.3 Relevant findings 

Statistical neighbour methodology sits within the wider area of performance 
assessment. A set of desirable characteristics for performance indicators was 
developed by Bird et al . , (2005). Amongst these was the importance of ‘adjust- 
ing for context to achieve comparability’ (Bird et al., 2005, p. 13). It stated that 
this process of adjustment ‘must be transparent to be convincing to the user’ 
(Bird et al., 2005, p. 13). 
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The development of statistical neighbours certainly has the potential to meet 
this criterion. However, the choice of contextual variables used to define statis- 
tical neighbours is clearly key: 

A clear issue is the choice of context variables on which to base adjustment. 
They should represent external features outside the control, judgement or influ- 
ence of the organization under assessment. This can be difficult to determine 
without local knowledge or audit. 

(Bird et al., 2005, p. 15) 

The only previous published methodology for defining statistical neighbours 
for benchmarking that was obtained through the literature searches, related to 
the work of Ofsted. Statistical neighbours were defined by a three stage 
process of: 

• proposing a large number of variables that could be used to define neighbours 

• removing those variables that were highly correlated with other possible vari- 
ables or did not correlate well with educational outcomes 

• defining statistical neighbours using the remaining variables giving equal 
weight to each one. 

No justification was made for giving equal weight to each of the remaining 
variables. 

It should be noted that, although there is relatively little published work on the 
use of statistical neighbours within benchmarking and performance assessment 
literature, statistical neighbours is a well established technique within the area 
of statistical prediction models. Within this framework statistical neighbour 
models are designed as a technique for predicting the likely behaviour of a new 
individual or observation and have been used in a wide range of medical and 
commercial applications. A summary of the research in this area can be found in 
Dasarathy (1991). 

One of the common features of these models is the optimisation of model 
parameters (such as the amount of weight given to different variables) with 
regard to the performance of the model. There are number of methods suggested 
for weighting variables including those suggested by Short and Fukunaga 
(1981), and Paredes and Vidal (2000). Although the new statistical neighbour 
model developed by NFER is not primarily intended for predictive purposes, 
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the principle of choosing model parameters to maximise the performance of the 
model can be extended to a benchmarking situation. 

Although, as noted above, there has been little published work in the area of 
choosing statistical neighbours for benchmarking, there has been much work 
published in the related area of deprivation indices. The most obviously applica- 
ble and widely used of the reports are the definitions of multiple deprivation 
indices as reported by the Office for the Deputy Prime Minister (2004). This 
work identified a number of different aspects of deprivation including income, 
employment, health, education, barriers to housing and services, living environ- 
ment and crime. A total of 37 indicators were used to measure deprivation 
across these seven domains. Shrinkage (that is, adjusting the values of indica- 
tors within small areas towards the average level for these indicators to improve 
accuracy) was applied to some indicators as part of this process. The resulting 
seven indices of deprivation were combined to form a single measure by rank- 
ing areas, using these ranks to transform data to the exponential distribution 
and summing the resulting seven numbers. As part of this process more weight 
was given to the income and employment domains and less to the domains of 
barriers to housing and services, crime, and living environment. No empirical 
justifications were made for this decision. 

Some of the more technical aspects of this work were commented on in the 
evaluation of Scottish indices of deprivation by Weir and McConnachie (2005). 
With regard to the issue of shrinkage (adjusting the values of indicators within 
small areas towards the average level for these indicators), they commented 
that: 

It has little effect on the resultant indices and by shrinkage towards LA aver- 
ages, introduces a small bias that penalises data zones within otherwise less 
deprived areas 

(Weir and McConnachie, 2005, p. 57) 

In terms of the relative weights given to different indicators the authors recom- 
mend that: 

the methods by which domain weights are derived are made more explicit 

reflecting the importance of each domain in terms of its prevalence and its 

severity 

(Weir and McConnachie, 2005, p. 57) 
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The practice of transforming indicators to the exponential distribution before 
combining them into single measures was generally supported as it reduced the 
incidence of low scores on one indicator, cancelling out high levels of depriva- 
tion on another. 

Concerns about the usefulness of many deprivation indices within rural areas were 
raised by Higgs and White (2000). They noted that certain indicators of deprivation 
were not appropriate for rural areas. For example in the case of car ownership: 

The necessity for a car in rural areas in the face of costly, or non-existent pub- 
lic transport services leads to higher than average levels of car ownership, 
often at the expense of low-income households not being able to afford other- 
types of household goods. 

(Higgs and White, 2000, p. 15) 

As a replacement for these types of indicators the authors recommended that 
indicators of the availability of services to residents are used. 

Questions about whether various deprivation indices had equal effects on men 
and women were raised by Stafford et ah, (2005). However, since the present 
study is dealing with relatively large population areas, gender imbalance is 
unlikely to be a problem and these issues can be safely ignored. 



A3.4 Conclusions from the literature review 

From the above literature, the following points were taken forward when devel- 
oping the methodology for the children’s services model. 

• Adjustment context is an important part of performance assessment. This 
could be achieved by the use of statistical neighbours. The methods used to 
define these need to be transparent. 

• Variables used in the development of statistical neighbours should only 
include those variables which are clearly beyond the control of children’s 
services at the local level. 

• The model devised by Ofsted provides a good basis upon which to base this 
work, although it would be good to have a clearer justification for the amount 
of weight given to different variables. Weighting variables to optimise model 
performance is common practice within statistical prediction literature and 
may provide a method for approaching this. 
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• Consideration needs to be given to the effectiveness of deprivation indices 
within areas containing large rural communities. 

• Information about availability of services should be considered as part of the 
statistical neighbour model. 

• It may be worth transforming indicators to the exponential distribution if 
these are to be combined to form summary measures prior to use in the statis- 
tical neighbours model. 
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Appendix 4 Criteria for comparing 
models 



A4.1 Introduction 

It is clear from reviewing current practice that a number of statistical neighbour 
models are available for, and in use in, local authorities in England. Each of these 
models makes use of different sets of data, combined in different ways to pro- 
duce neighbours. Whilst some decisions about the ways statistical neighbours are 
defined are justified empirically (such as using the correlations of various socio- 
economic and demographic variables with outcomes to determine which 
variables should be included in the model) there are other areas where the deci- 
sions made appear arbitrary. For example, both the Institute of Public Finance 
comparator councils and Ofsted statistical neighbour models give equal weight 
to all background variables. Currently there is no empirical justification for this 
practice. Similarly the Ofsted model provides local authorities with ten or fewer 
statistical neighbours whilst the IPF model provides 15. Once again, there is cur- 
rently no empirical justification for the differences between these models. 

In order to address this situation, and to base any future statistical neighbour 
model on a more empirical basis it is necessary to develop a method for compar- 
ing the effectiveness of one statistical neighbour model with another. A possible 
criterion for doing this is developed below. 

It is important to note that the criterion developed below is not intended as a cri- 
terion for defining statistical neighbours in and of itself. The aim here is to 
produce an empirical measure for comparing different methodologies so that if 
we are presented with two differing methodologies for producing statistical 
neighbours we can tell which of these is more appropriate in our given context. 
The validity of this criterion is dependent upon the purpose for which statistical 
neighbours are to be used. Therefore, the criterion derived below may not be 
appropriate in other contexts. 

A4.2 A single assessment criterion 

In order to devise a criterion for assessing statistical neighbour methodologies it 
is important to keep in mind the main purpose for which these models are used. 
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Statistical neighbours are intended to provide an indication of whether local 
authorities are performing above or below the level that might be expected. For 
this reason, it is our opinion that it is necessary that the performances of a local 
authority’s statistical neighbours provide a reasonable indicator of what might 
be expected from that local authority. In other words, a good statistical neigh- 
bour model should be such that the expected performance of each local 
authority is close to the expected average performance of its statistical neigh- 
bours. This rationale for assessing the performance of a statistical neighbour 
model can be realised in terms of a quantitative criterion. In theory we might 
measure this by the average squared difference between the expected perform- 
ance of each local authority and the average expected performance of its 
statistical neighbours. 

In practice, unfortunately, this approach cannot be applied directly because the 
precise relationship between the demographics of a particular local authority 
and the expected value of each performance indicator is not known. However, it 
has been shown in other contexts that there is a close relationship between the 
differences in the expected values of outcomes within a given grouping of indi- 
vidual units and the differences between the actual values of outcomes (Benton 
and Hand, 2002). It is reasonable to assume that this finding would equally 
apply to statistical neighbour models. As a result we can infer that the model 
that produces statistical neighbours such that the performance of each local 
authority is closest to the average actual performance of its statistical neigh- 
bours, is the model that is performing best in terms of matching the expected 
values of performance. 

Using this logic, the effectiveness of any statistical neighbour model can be 
assessed using a simple formula. Suppose we have J separate performance indi- 
cators for each of N local authorities. In the context of providing statistical 
neighbours appropriate for children and young people’s services these might be 
indicators such as the rate of conceptions amongst young people under the age 
of 1 8 within the local authority, or the total number of half days absence from 
school per pupil of statutory age. In order for differences in performance on dif- 
ferent indicators to be meaningfully compared, it is important that these 
outcomes are standardised. Let us denote the standardised performance of the 
ith local authority on the / 1 h performance indicator as I ij . Now we can define our 
measure as: 

Measure - (X„ (T ff - Up 2 ) NJ 
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Where: 



Y t] - Standardised performance of LA i on performance indicator j 

Yy = Average standardised performance of statistical neighbours of LA i on 
performance indicator j for the given statistical neighbour model 

J = Number of performance indicators 

N - Number of local authorities 

In plain language our measure is the average squared difference between the 
performance of each local authority and the average performance of its statisti- 
cal neighbours across all performance indicators. If presented with two different 
statistical neighbour models we now choose the model that results in the lowest 
value for this measure. 

Aside from the justification given above, this criterion has some face validity. 
Generally speaking, if the performances of individual local authorities were 
closer to national averages than to the average for their statistical neighbours, 
the statistical neighbour method of benchmarking would be at best inappropri- 
ate and at worst misleading. There would be little point in comparisons to 
statistical neighbours being made if we did not expect there to be similarities in 
outcomes. 

In some ways using this criterion to define the parameters of a statistical neigh- 
bour model has similarities to the value added approach used for assessing 
schools. Each school’s performance is compared to their expected performance 
given their prior attainment. The expected performance of each school is 
defined in terms of the prior attainment of their pupils such that nationally 
school performance tends to be close to the expected performance. Within this 
framework, individual schools may differ substantially from their expected per- 
formance but nationally most schools will be close. In other words the standard 
value added methodology defines the parameters of its underlying model such 
that nationally, performance is close to what was expected from the model. This 
same idea can be applied to local authorities in the way the criterion described 
is used to define statistical neighbours. 

Once again it is important to reiterate that this criterion cannot be used to 
directly produce statistical neighbours. The statistical neighbours of each local 
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authority are defined in terms of various demographic and socio-economic indi- 
cators as in the various existing models. The idea here is to provide a criterion 
for comparing different methodologies for compiling lists of statistical neigh- 
bours rather than to produce a particular methodology. 

As such, although under this approach the effectiveness of any statistical neigh- 
bour model is assessed in terms of performance indicators, statistical 
neighbours themselves are not defined by performance indicators. That is, sta- 
tistical neighbours are defined by a selection of demographic and 
socio-economic variables as in the Ofsted and IPF models. However, using this 
criterion, it is possible to compare models that include different combinations of 
variables and that give different amounts of weight to different variables. 
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Appendix 5 Defining statistical 
neighbours 



The first step in defining statistical neighbours was determining which vari- 
ables should be used in the model. This process began with a list of 63 potential 
background indicators describing the characteristics of each local authority. 
This list of suggested variables was informed by the consultation with local 
authorities and other stakeholders, the literature review and examining which 
variables are used in existing statistical neighbour models. 

Potential background variables were collated from the following sources: 

• The 2001 census 

• Annual population surveys between 2001 and 2005 

• Labour force survey four quarterly averages - June 2004 to May 2005 

• Annual survey of hours and earnings 2005 

• The ODPM indices of multiple deprivation 

• The local authority data matrix 

• DVLA information on vehicle numbers and ages 

• CIPFA information on availability of services 

One important decision made at this stage was the decision not to use percent- 
age of students with a statement of special educational needs as a potential 
indicator as policies on these varied too greatly between local authorities. 

In order to reduce this list of 63 to a more manageable amount of data and pro- 
duce the final definitions for statistical neighbours, a four step process was used. 

1. Regression analysis of performance indicators on the proposed background 
variables. This process was used to reveal which performance indicators were 
appropriate for benchmarking using a statistical neighbour model and which 
background variables had little or no association with performance and could be 
discarded. 

2. Analysis of correlations between background variables. Where high correla- 
tions existed between two variables, one of these was removed. 



the development of the Children’s Services Statistical Neighbour Benchmarking Model 



53 



3 . 



Calculating the appropriate number of statistical neighbours to assign to each 
local authority. 

4. Assigning weights to each of the remaining background variables in order to 
define statistical neighbours. 

A5.1 Regression analyses 

Of particular interest was ascertaining which of the many variables that had 
been proposed were important in determining what we might expect the level of 
the Every Child Matters performance indicators to be for each authority. In 
order to explore this, data was collected for the following indicators: 

• Infant mortality rate 

• Under 18 conception rate 

• Re-registrations on the Child Protection Register 

• % of children who have been looked after for more than two and a half years 
and of those, have been in the same placement for at least two years or placed 
for adoption 

• Number of 0-15 year olds injured or killed in road traffic accidents 

• Half days missed through absence 

• % of 7 year olds achieving L2+ at KS 1 

• % of 1 1 year olds achieving L4+ in English, Maths & Science 

• % of 14 year olds achieving L5+ in English, Maths & Science 

• % of 16 year olds achieving the equivalent of 5 A*-C at GCSE 

• % of pupils permanently excluded 

• % of pupils who had 1 or more episodes of fixed period exclusion 

• % of 10-17 year olds living in the local police force area who had been given 
a final warning / reprimand / caution 

• % of 16-18 year olds not in education, employment and training 

A stepwise regression procedure of each of the above indicators on all 63 proposed 
background characteristics was performed. This process fulfilled two purposes. 
Firstly this process revealed three outcomes that had extremely weak relationships 
with the proposed background variables. The following outcomes all had an 
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adjusted R2 coefficient of around 0. 1 or less (that is, less than ten per cent of the 
variation in these outcomes could be explained by the background variables): 

• Re-registrations on the Child Protection Register 

• % of children who have been looked after for more than two and a half years 
and of those, have been in the same placement for at least two years or placed 
for adoption 

• % of pupils permanently excluded 

As a result, it is recommended that statistical neighbour methodology is not 
used to benchmark performance on these indicators. These indicators were not 
included in any further analysis. 

Regression of all other performance indicators on the proposed background vari- 
ables led to an adjusted R2 coefficient of around 0.3 or greater. Using the results 
of the regression analyses for these outcomes, all proposed background indicators 
were ranked by their largest standardised coefficient in any regression. A number 
of variables with small values of largest standardised coefficients were removed 
from further analysis. Some variables defining the ethnic make up of the local 
authority were kept for further analysis despite having small largest standardised 
coefficients since they were known to be of particular interest to practitioners. 
There were 43 remaining potential background variables after this process. 

A5.2 Analysis of correlations 

Amongst the remaining variables, pairwise correlations were examined in order 
to discover which variables were highly correlated with other potential vari- 
ables. Where two variables were found to have a correlation of 0.9 or higher one 
of these variables was selected for further analysis based on the following three 
criteria which were applied in order. 

1 . Simple measures (i.e. simple percentages) were preferred to measures that were 
the result of complex manipulation (such as the indices of deprivation derived 
by the ODPM) since these would be easier to update in future. 

2. Measures that were updated regularly were preferred to census variables that are 
only updated once a decade. 

3. If the other two criteria did not separate the variables, the variable with the higher 
largest standardised coefficient across the regression analyses was chosen. 
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For reasons of interpretation a few additional variables were removed at this 
stage if they were deemed to have a meaning sufficiently similar to the meaning 
of another remaining variable to cause confusion. Each of these variables had a 
correlation with one of the other remaining variables of just less than 0.9. This 
process reduced the number of potential variables to 25. These variables were 
analysed further to assign appropriate weights and are listed Table A5. 1 . 



Table A5.1 Variables taken into weighting process 



Variables (25) Source 



Mean Weekly pay - gross 
Population density (per km2) 

% of the population living in villages, hamlets or isolated settlements 

% dependent Children - Occupancy Rating - +2 or more 

% dependent children living in overcrowded households 

% dependent Children - Two or more cars or vans in households 

% dependent children not in a family 

% dependent children in one adult households 

% dependent children in households where HRP is in any 
managerial and professional occupations 

% dependent children in households where HRP is in routine 
occupations 

% people with mixed ethnicity 
% people with Indian ethnicity 
% people with Pakistani ethnicity 
% people with Bangladeshi ethnicity 
% people with other Asian ethnicity 
% people with Black Caribbean ethnicity 
% people with Black African ethnicity 
% people with other Black ethnicity 
% people in good health 

% of working age people with higher qualifications 

% households owned outright or with mortgage 

% of households with dependent children living at same address 
one year ago 

% households with 3 or more dependent children 
% know to be eligible for FSM (2005) 

% vehicles that are 3 years old or less 



Annual Survey of Hours 
and Earnings 2005 

Annual Population Survey 2004 

2001 Census 

2001 Census 

2001 Census 

2001 Census 

2001 Census 

2001 Census 

2001 Census 

2001 Census 

2001 Census 
2001 Census 
2001 Census 
2001 Census 
2001 Census 
2001 Census 
2001 Census 
2001 Census 
2001 Census 
2001 Census 
2001 Census 
2001 Census 

2001 Census 
DfES (NPD 2005) 

DVLA 2004 
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A5.3 Criterion for choosing number of neighbours for 
each local authority and the weight given to 
each variable 

In order to tackle these issues, use was made of a criterion for comparing statis- 
tical neighbours (defined in appendix 4). This criterion was defined as the 
average squared difference between the performance of each local authority and 
the mean performance of its statistical neighbours across all outcomes and local 
authorities. The basis of this criterion is that statistical neighbour models are pri- 
marily designed to provide a reasonable benchmark for the performance of local 
authorities. For this reason it desirable that, generally speaking, the performance 
of local authorities is close to the expected performance derived from their sta- 
tistical neighbours. This does not preclude the possibility of individual local 
authorities having a performance very different from that of their statistical 
neighbours, but does ensure that this occurrence is not the norm. 

A5.4 How many statistical neighbours to assign to 
each local authority 

Using the criterion defined in appendix 4, it was possible to compare the per- 
formance of statistical neighbour models which assigned different numbers of 
neighbours to each local authority. This analysis was performed on the basis that 
each variable in the model had equal weight. The results are shown in the table 
A5.2. It should be noted that all outcome variables have been scaled such that 
the criterion has a value of one, when we compare each local authority’s per- 
formance to the national mean. Hence the figures essentially show the average 
squared difference between each LA’s performance and that of its statistical 
neighbours as a percentage of the averaged squared difference between each 
LAs performance and the national mean. 

It can be seen from the table that the differences between a local authority’s per- 
formance and that of its statistical neighbours tend to be substantially less than 
differences with the national mean, regardless of the number of neighbours cho- 
sen. This fact provides some validation for the idea of using statistical 
neighbours to provide context adjustment as a whole. 
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Table A5.2 Exploring different numbers of statistical neighbours 



Number of neighbours assigned 
to each local authority 


Value of criterion 


2 


0.84 


5 


0.71 


10 


0.68 


20 


0.69 


148 (Comparison to national mean) 


1.00 



Note: Outcome variables scaled such that each of the five Every Child Matters Outcomes were given roughly equal weight. 
Variables scaled so that criterion has a value of 1 when we compare each local authority's performance to the national mean. 



As can be seen from the table, the optimal performance was achieved by assign- 
ing ten statistical neighbours to each local authority. In this case the average 
squared difference in performance between each LA and that of its statistical 
neighbours is just 68 per cent of the difference between each LA’s performance 
and the national mean. For this reason, further analysis assumed that each local 
authority would be assigned ten neighbours. 

A5.5 Weighting variables 

An important decision at this stage was how much importance each variable 
should be assigned in the statistical neighbour model relative to other variables. 
A search of many possible sets of weights was performed to minimise the crite- 
rion defined above. This analysis revealed that three of the variables that were 
examined in this process should be assigned a weight of zero or close to zero. 
These variables were removed from the statistical neighbour model and the final 
weight assigned to each variable is shown in table A5.3. 

These weights were used to define the final model. The performance of this 
model is compared to the performance of a model assigning equal weights to all 
variables as well as to the performance of Ofsted’s statistical neighbours and to 
the Institute of Public Finance comparator councils. The results of this compar- 
ison are shown in table A5.4. 
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Table A5.3: Weight assigned to each variable 



Variable Weight 

Mean Weekly pay - gross 3 

% pupils known to be eligible for FSM (2005) 17 

% vehicles that are 3 years old or less 1 5 

% dependent Children - Occupancy Rating - +2 or more 4 

% dependent children living in overcrowded households 19 

% dependent Children - Two or more cars or vans in households 8 

% dependent children in one adult households 1 2 

% dependent children in households where HRP is in any 5 

managerial & professional occupations 

% dependent children in households where HRP is in routine 9 

occupations 

% people with mixed ethnicity 5 

% people with Indian ethnicity 9 

% people with Pakistani ethnicity 2 

% people with Bangladeshi ethnicity 7 

% people with other Asian ethnicity 1 6 

% people with Black Caribbean ethnicity 30 

% people with Black African ethnicity 1 4 

% people with other Black ethnicity 8 

% of working age people with higher qualifications 27 

% people in good health 6 

% households owned outright or with mortgage 24 

% households with 3 or more dependent children 9 

% of the population living in villages, hamlets or isolated 4 

settlements 

Population density (per km2) Removed 

% dependent children not in a family Removed 

% of households with dependent children living at same address Removed 

one year ago 
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Table A5.4 Relative performance of different statistical neighbour models 



Statistical neighbour model 


Value of criterion 


Final Model 


0.63 


Equal Weights Model 


0.68 


Ofsted’s Statistical Neighbours 


0.72 


Institute of Public Finance comparator councils 


0.71 


Comparison to national means 


1.00 



Note: Outcome variables scaled such that each of the five Every Child Matters Outcomes were given roughly equal weight. 
Variables scaled so that criterion has a value of 1 when we compare each local authority's performance to the national mean. 



A5.6 Further validation 

Although it has been shown above that the newly derived statistical neighbour 
model provides a reasonable benchmark when we average this performance 
looking across different outcome indicators, it is clearly desirable that the model 
provides a sensible benchmark not just on average but across all potential out- 
come indicators. This issue is explored in table A5.5. The table shows the 
average squared difference between the performance of each local authority on 
each outcome indicator and the average performance of its statistical neigh- 
bours. 

A number of points can be noted from this table. Firstly it can be seen that for 
each model, the outcomes removed from analysis earlier (see section A5. 1) are 
not well predicted by statistical neighbours. In each case local authorities’ per- 
formances on these indicators tend to be closer to the national mean than to the 
average performance of their statistical neighbours. For all other outcomes, with 
the exception of two, the new proposed model provides benchmarks for local 
authorities that are at least as accurate as any other statistical neighbour model. 
The two exceptions are in the cases of key stage 1 writing (where the perform- 
ance of the equal weights model is slightly superior) and key stage 1 maths 
(where Institute of Public Finance comparator councils appear to be unusually 
close). 

From this evidence it can be assumed that the statistical neighbour model 
derived here will provide a reasonable benchmark for performance across most 
Every Child Matters outcomes. 
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Table A5.5 Mean square error of each model in assigning expected performance for each 
outcome indicator 



Variable 


Final 

Model 


Equal 

Weights 

Model 


Ofsted’s Institute of 

Statistical Public 
Neighbours Finance 

comparator 

councils 


Infant mortality rate: deaths up to 1 year per 
1,000 live births 


0.74 


0.81 


0.87 


0.87 


U18 Conception Rate 2004 


0.39 


0.45 


0.50 


0.56 


Percentage of children who have been looked 
after for more than two and a half years and of 
those, have been in the same placement for at 
least two years or placed for adoption* 


1.11 


1.09 


1.09 


1.04 


Percentage of children registered on the 
child protection register, previously registered 
2004/05* 


1.11 


1.09 


1.03 


1.02 


Total number of child casualties in road 
accidents per 1000 U16s (2003) 


0.74 


0.81 


0.81 


0.74 


KS1 % L2 Reading 


0.38 


0.38 


0.45 


0.45 


KS1 % L2 Writing 


0.45 


0.42 


0.49 


0.49 


KS1 % L2 Maths 


0.45 


0.44 


0.57 


0.37 


KS2 % L4 English 


0.40 


0.40 


0.50 


0.50 


KS2 % L4 Maths 


0.46 


0.47 


0.57 


0.60 


KS2 % L4 Science 


0.46 


0.47 


0.59 


0.48 


KS3 % L5 English 


0.34 


0.36 


0.41 


0.43 


KS3 % L5 Maths 


0.32 


0.34 


0.34 


0.40 


KS3 % L5 Science 


0.30 


0.31 


0.31 


0.37 


KS4 % 5+ A*-C 


0.56 


0.57 


0.57 


0.66 


Total absence rate at primary schools 2004/05 


0.40 


0.40 


0.53 


0.42 


Total absence rate at secondary schools 2004/05 


0.57 


0.59 


0.62 


0.58 


Percentage of the total school population 
permanently excluded* 


1.08 


1.00 


1.02 


1.04 


Percentage of pupils who had 1 or more 
episodes of fixed period exclusion as a 
percentage of the school population 


0.85 


0.89 


1.00 


0.98 


The % of all children aged 10-17 living in the 
local police force area who had been given a 
final warning / reprimand / caution or convicted 
during the previous calendar year 


0.64 


0.71 


0.70 


0.85 


% of 16 & 17 year olds in education and WBL 


0.70 


0.76 


0.73 


0.78 



*Variable not used in process of assigning weights. 

Note: Variables scaled so that mean square error has a value of 1 when we compare each local authority's performance to the national mean. 
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Appendix 6 Screenshot of the 
new model 



Children’s Services Statistical Neighbour Benchmarking Tool 



(Highlighted boxes c an be edited) 





Select Local Authority Indicator 1 


Indicator 2 


fri dicator 3 


Indicator 4 


fri dicator 5 


Infant mortality rate: 
deaths up to 1 year per 

Barking & Dagenham 1 ,000 live births (2005) 

3.70 


U18 Conception Rate 
(2004) 
71.80 


5* A*-C inc. Engliii & 
Mathematics (2006) 
37.60 


Total absence rate at 
secondary schools 
(2005/06) 
8.12 


The % of all children 
aged 10-17 living in 
the local police force 
area who had been 
given a final warning / 
reprimand / caution or 
convicted during 
twelve monthsending 
30 September 2005 
3.25 


Average for statistical neighbours 8.56 


54.42 


33.27 


9.02 


3.99 












Closest Demographic Neighbours 

Infant mortality rate: 
deaths up to 1 year per 

Rank (1=Closest) Name "Closeness” 1 ,000 live births (2005) 


U18 Conception Rate 
(2004) 


5* A*-C inc . English & 
Mathematics (2006) 


Total absence rate at 
secondary schools 
12005/06) 


The % of all children 
aged 10-17 living in 
the local police force 
area who had been 
given a final warning / 
reprimand / caution or 
convicted during 
twelve monthsending 
30 September 2005 


Nottingham City jCtose f 7.50 


72.80 


28.10 


9.87 


4.19 


2 Sandwell 'Close 7.70 


56.60 


29.60 


9.72 


4.10 


3 Greenwich iCIose f 3.80 


64.70 


31.30 


9.33 


3.25 


4 Southampton \ Close ' 5.00 


56.10 


35.90 


8.96 





5 _ Hull 1 Close ? 4.40 


25.60 


25.80 


10.74 


4.35 


6 Middlesbrough 'Close ^ 7.80 


61.60 


30.10 


10.12 


4.53 


Birmingham iCIose f 7.80 


51.50 


40.70 


7.69 


4.10 


8 Walsall [base T 10-20 


48.70 


34.80 


7.99 


4.10 


9 Coventry jCtose > 4.40 


52.80 


37.50 


8.24 


4.10 


10 Peterborough City 'Close ^ 7.00 


53.80 


38.90 


7.50 


2.74 



Wole on distance measure 

The distance between any two local authonbes is defined as the weighted Euclidean distance 
between the authorities usng each of the background variables. 'Closeness as displayed in the 
above table is defined as follows: 

Extremely Close: Weighted Euclidean distance between local authorities is equivalent to less than 

Very Close: Weighted Euclidean distance between local authorities is equivalent to less than 

Close: Weighted Euclidean distance between local authorities is equivalent to less than 

Somewhat Close: Weighted Euclidean distance between local authorities is equivalent to less than 

Not Close: Weighted Euclidean distance between local authorities is equivalent to 1.15 per 

Fora description of Euclidean distance see the analytical user guide. 



0.25 per standardised vanable 
0.55 per standardised variable 
0.85 per standardised variable 
1.15 per standardised variable 
standardised variable or more 



©NFER 



62 



the development of the Children’s Services Statistical Neighbour Benchmarking Model 



Appendix 7 List of local 
authorities’ statistical neighbours 



Each local authority’s statistical neighbours are listed below. Local authorities 
are listed in alphabetical order. The ten statistical neighbours for each authority 
are listed with the closest statistical neighbour first and the most different listed 
last. 

BARKING - Nottingham City, Sandwell, Greenwich, Southampton, Hull, Mid- 
dlesbrough, Birmingham, Walsall, Coventry, Peterborough City. 

BARNET - Kingston on Thames, Merton, Hillingdon, Reading, Sutton, Houn- 
slow, Redbridge, Bromley, Richmond upon Thames, Enfield. 

BARNSLEY - Hartlepool, Rotherham, Redcar & Cleveland, Wakefield, Don- 
caster, Sunderland, Durham, St Helens, Stoke on Trent, Gateshead. 

BATH & NORTH EAST SOMERSET - Gloucestershire, Cambridgeshire, 
Oxfordshire, West Sussex, Hampshire, Wiltshire, North Somerset, Shropshire, 
Dorset, East Sussex. 

BEDFORDSHIRE - Hertfordshire, Hampshire, Essex, Warwickshire, Solihull, 
Kent, Bracknell Forest, Cheshire, West Berkshire, Worcestershire. 

BEXLEY - Havering, Medway, Swindon, Essex, Kent, Thurrock, Southend, 
Stockport, Sutton, Bedfordshire. 

BIRMINGHAM - Luton, Sandwell, Wolverhampton, Nottingham City, Bark- 
ing, Coventry, Enfield, Bradford, City of Derby, Walsall. 

BLACKBURN - Bradford, Walsall, Coventry, Rochdale, Bolton, Oldham, 
Kirklees, City of Derby, Peterborough City, Middlesbrough. 

BLACKPOOL - Torbay, Tameside, Redcar & Cleveland, Plymouth, Hartle- 
pool, North East Lincolnshire, Barnsley, Isle of Wight, Doncaster, Stoke on 
Trent. 

BOLTON - Kirklees, Coventry, Calderdale, Stockton on Tees, Tameside, City 
of Derby, Leeds, Dudley, St Helens, Telford & Wrekin. 
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BOURNEMOUTH - Southend, Torbay, Isle of Wight, Portsmouth, Poole, East 
Sussex, Plymouth, Swindon, Devon, Kent. 

BRACKNELL FOREST - Hertfordshire, West Berkshire, Buckinghamshire, 
Hampshire, Surrey, Bedfordshire, Windsor & Maidenhead, Oxfordshire, Cam- 
bridgeshire, Cheshire. 

BRADFORD - Rochdale, Oldham, Blackburn, Kirklees, Walsall, Bolton, City 
of Derby, Coventry, Calderdale, Peterborough City. 

BRENT - Ealing, Waltham Forest, Haringey, Croydon, Lewisham, Newham, 
Enfield, Merton, Harrow, Redbridge. 

BRIGHTON & HOVE - Bristol, Bournemouth, Portsmouth, Reading, 
Sheffield, Southampton, Bath & North East Somerset, Southend, City of York, 
Plymouth. 

BRISTOL - Southampton, Portsmouth, Brighton & Hove, Reading, City of 
Derby, Peterborough City, Sheffield, Plymouth, Bournemouth, Southend. 

BROMLEY - Hertfordshire, Trafford, Sutton, Stockport, Bedfordshire, Brack- 
nell Forest, Solihull, West Sussex, Hampshire, Bath & North East Somerset. 

BUCKINGHAMSHIRE - Surrey, West Berkshire, Bracknell Forest, Windsor & 
Maidenhead, Hertfordshire, Oxfordshire, Bedfordshire, Hampshire, Cam- 
bridgeshire, Wokingham. 

BURY - Lancashire, Stockton on Tees, Calderdale, Sefton, Stockport, Warring- 
ton, Nottinghamshire, Darlington, Solihull, Bolton. 

CALDERDALE - Lancashire, Bury, Stockton on Tees, Kirklees, Bolton, Dud- 
ley, Darlington, St Helens, Nottinghamshire, Leeds. 

CAMBRIDGESHIRE - Oxfordshire, Bath & North East Somerset, Wiltshire, 
Hampshire, Gloucestershire, West Berkshire, Hertfordshire, Worcestershire, 
Bedfordshire, West Sussex. 

CAMDEN - Westminster, Hammersmith & Fulham, Islington, Kensington & 
Chelsea, Wandsworth, Greenwich, Haringey, City of London, Manchester, Bar- 
net. 
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CHESHIRE - Warwickshire, Warrington, Stockport, Hampshire, Solihull, 
Worcestershire, Bedfordshire, North Yorkshire, East Riding of Yorks, Stafford- 
shire. 

CITY OF DERBY - Peterborough City, Coventry, Telford & Wrekin, Bolton, 
Sheffield, Walsall, Leeds, Portsmouth, Dudley, Tameside. 

CITY OF LONDON - Kensington & Chelsea, Westminster, Camden, 
Wandsworth, Hammersmith & Fulham, Richmond upon Thames, Islington, 
Brighton & Hove, Barnet, Newcastle upon Tyne. 

CITY OF YORK - Stockport, Warrington, Cheshire, Nottinghamshire, War- 
wickshire, Trafford, Staffordshire, Bury, Swindon, Lancashire. 

CORNWALL - Norfolk, Devon, Somerset, Herefordshire, Suffolk, Shropshire, 
Cumbria, Lincolnshire, Dorset, Isle of Wight. 

COVENTRY - City of Derby, Bolton, Walsall, Peterborough City, Telford & 
Wrekin, Tameside, Kirklees, Leeds, Sheffield, Thurrock. 

CROYDON - Enfield, Waltham Forest, Redbridge, Merton, Birmingham, Eal- 
ing, Reading, Hillingdon, Luton, Greenwich. 

CUMBRIA - Derbyshire, Lincolnshire, Norfolk, Nottinghamshire, Stafford- 
shire, Lancashire, Devon, Cornwall, North Lincolnshire, Shropshire. 

DARLINGTON - Stockton on Tees, North Tyneside, St Helens, Durham, 
Wigan, Bury, Lancashire, Tameside, Halton, Calderdale. 

DERBYSHIRE - Nottinghamshire, Staffordshire, Cumbria, Lincolnshire, Lan- 
cashire, Wigan, Northamptonshire, Dudley, Kent, Warwickshire. 

DEVON - Somerset, Shropshire, Suffolk, Cornwall, Dorset, Herefordshire, 
Gloucestershire, Norfolk, East Sussex, Worcestershire. 

DONCASTER - Rotherham, Redcar & Cleveland, Wigan, St Helens, Tame- 
side, Wakefield, Dudley, North East Lincolnshire, Barnsley, North Lincolnshire. 

DORSET - Shropshire, Devon, Gloucestershire, Worcestershire, West Sussex, 
Somerset, East Sussex, Suffolk, North Somerset, Poole. 
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DUDLEY - Wigan, Doncaster, Nottinghamshire, Thurrock, Rotherham, Lan- 
cashire, Derbyshire, Telford & Wrekin, Stockton on Tees, Bolton. 

DURHAM - Wakefield, North Tyneside, Sunderland, St Helens, Darlington, 
Gateshead, Barnsley, Stockton on Tees, Wigan, Doncaster. 

EALING - Hounslow, Merton, Harrow, Redbridge, Enfield, Barnet, Slough, 
Croydon, Brent, Hillingdon. 

EAST RIDING OF YORKS - North Yorkshire, Staffordshire, Warwickshire, 
Cheshire, Worcestershire, Warrington, Derbyshire, Nottinghamshire, Leicester- 
shire, Essex. 

EAST SUSSEX - Kent, Dorset, West Sussex, Gloucestershire, Devon, Essex, 
North Somerset, Worcestershire, Shropshire, Suffolk. 

ENFIELD - Croydon, Hillingdon, Redbridge, Waltham Forest, Reading, Mer- 
ton, Luton, Birmingham, Sutton, Barnet. 

ESSEX - Kent, West Sussex, Hampshire, Worcestershire, Bedfordshire, Swin- 
don, North Somerset, Warwickshire, Staffordshire, Leicestershire. 

GATESHEAD - Sunderland, South Tyneside, Durham, North Tyneside, Barns- 
ley, Wakefield, Hartlepool, St Helens, Halton, Salford. 

GLOUCESTERSHIRE - Shropshire, Dorset, Bath & North East Somerset, 
Wiltshire, West Sussex, Suffolk, Devon, Worcestershire, Somerset, Cam- 
bridgeshire. 

GREENWICH - Barking, Manchester, Nottingham City, Enfield, Birmingham, 
Waltham Forest, Islington, Bristol, Southampton, Hammersmith & Fulham. 

HACKNEY - Southwark, Lambeth, Haringey, Lewisham, Newham, Islington, 
Waltham Forest, Hammersmith & Fulham, Greenwich, Brent. 

HALTON - Hartlepool, St Helens, Tameside, Redcar & Cleveland, Sunderland, 
Stockton on Tees, Darlington, North East Lincolnshire, Salford, South Tyneside. 

HAMMERSMITH & FULHAM - Wandsworth, Islington, Westminster, Cam- 
den, Kensington & Chelsea, Haringey, Greenwich, Waltham Forest, Southwark, 
Lambeth. 



66 



the development of the Children’s Services Statistical Neighbour Benchmarking Model 



HAMPSHIRE - West Sussex, South Gloucestershire, Bedfordshire, North 
Somerset, Essex, Worcestershire, West Berkshire, Hertfordshire, Cam- 
bridgeshire, Leicestershire. 

HARINGEY - Lewisham, Lambeth, Hackney, Waltham Forest, Southwark, 
Islington, Hammersmith & Fulham, Wandsworth, Greenwich, Croydon. 

HARROW - Ealing, Redbridge, Merton, Hounslow, Barnet, Hillingdon, 
Kingston on Thames, Slough, Enfield, Croydon. 

HARTLEPOOL - Halton, Redcar & Cleveland, Barnsley, Tameside, Rother- 
ham, St Helens, Doncaster, North East Lincolnshire, Sunderland, South 
Tyneside. 

HAVERING - Bexley, Essex, Medway, Kent, Swindon, Thurrock, Bury, Soli- 
hull, Stockport, Staffordshire. 

HEREFORDSHIRE - Somerset, Devon, Shropshire, Cornwall, Suffolk, Nor- 
folk, Wiltshire, Dorset, Gloucestershire, East Sussex. 

HERTFORDSHIRE - Bracknell Forest, Bedfordshire, Hampshire, Oxfordshire, 
Cambridgeshire, West Berkshire, Buckinghamshire, Cheshire, Trafford, Surrey. 

HILLINGDON - Sutton, Coventry, Reading, Redbridge, Hounslow, Barnet, 
Bexley, Slough, City of Derby, Milton Keynes. 

HOUNSLOW - Ealing, Hillingdon, Slough, Redbridge, Barnet, Merton, 
Enfield, Harrow, Leicester, Reading. 

HULL - Stoke on Trent, Middlesbrough, Hartlepool, Salford, South Tyneside, 
Southampton, Plymouth, Barnsley, Rotherham, Halton. 

ISLE OF WIGHT - Torbay, Southend, Bournemouth, East Sussex, Cornwall, 
Devon, Norfolk, Cumbria, Nottinghamshire, Poole. 

ISLES OF SCILLY - Herefordshire, Cornwall, Wiltshire, Norfolk, Shropshire, 
Somerset, Devon, Suffolk, Rutland, Cambridgeshire. 

ISLINGTON - Hammersmith & Fulham, Camden, Westminster, Haringey, 
Greenwich, Southwark, Wandsworth, Manchester, Lambeth, Kensington & 
Chelsea. 
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KENSINGTON & CHELSEA - Westminster, Wandsworth, Hammersmith & 
Fulham, Camden, City of London, Islington, Barnet, Richmond upon Thames, 
Merton, Greenwich. 

KENT - Essex, Swindon, East Sussex, Northamptonshire, Worcestershire, War- 
wickshire, West Sussex, Staffordshire, Lancashire, Bedfordshire. 

KINGSTON ON THAMES - Barnet, Sutton, Merton, Richmond upon Thames, 
Bromley, Windsor & Maidenhead, Hillingdon, Hertfordshire, Surrey, Reading. 

KIRKLEES - Bolton, Calderdale, Rochdale, Bury, Leeds, Stockton on Tees, 
Oldham, Dudley, City of Derby, Lancashire. 

KNOWSLEY - Salford, Middlesbrough, Liverpool, South Tyneside, Halton, 
Hartlepool, Newcastle upon Tyne, Rochdale, Tameside, Gateshead. 

LAMBETH - Haringey, Lewisham, Southwark, Hackney, Islington, Hammer- 
smith & Fulham, Waltham Forest, Wandsworth, Greenwich, Brent. 

LANCASHIRE - Nottinghamshire, Bury, Calderdale, Sefton, Derbyshire, 
Stockton on Tees, Kent, Staffordshire, Northamptonshire, Dudley. 

LEEDS - Sheffield, Bolton, Stockton on Tees, Darlington, Calderdale, St 
Helens, Kirklees, City of Derby, North Tyneside, Bury. 

LEICESTER - Wolverhampton, Hounslow, Sandwell, Blackburn, Slough, 
Coventry, Hillingdon, Walsall, Birmingham, Southampton. 

LEICESTERSHIRE - South Gloucestershire, Warwickshire, Worcestershire, 
North Somerset, Hampshire, Staffordshire, Essex, West Sussex, Bedfordshire, 
Cheshire. 

LEWISHAM - Haringey, Lambeth, Waltham Forest, Hackney, Southwark, 
Croydon, Brent, Hammersmith & Fulham, Islington, Greenwich. 

LINCOLNSHIRE - Cumbria, Derbyshire, Norfolk, Suffolk, Shropshire, Not- 
tinghamshire, Worcestershire, Staffordshire, Northamptonshire, Dorset. 

LIVERPOOL - Salford, Knowsley, South Tyneside, Newcastle upon Tyne, 
Middlesbrough, Halton, Gateshead, Hartlepool, Sunderland, Hull. 
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LUTON - Birmingham, Bradford, Slough, Enfield, Coventry, Walsall, City of 
Derby, Sandwell, Blackburn, Peterborough City. 

MANCHESTER - Nottingham City, Greenwich, Liverpool, Barking, Birming- 
ham, Salford, Middlesbrough, Hull, Southampton, Newcastle upon Tyne. 

MEDWAY - Thurrock, Swindon, Kent, Bexley, Northamptonshire, Havering, 
Telford & Wrekin, Lancashire, Dudley, Southend. 

MERTON - Barnet, Kingston on Thames, Ealing, Redbridge, Enfield, Houn- 
slow, Hillingdon, Croydon, Reading, Sutton. 

MIDDLESBROUGH - Salford, Hartlepool, Rochdale, South Tyneside, Halton, 
Knowsley, Hull, Redcar & Cleveland, Stoke on Trent, Tameside. 

MILTON KEYNES - Northamptonshire, Trafford, Swindon, Kent, Bedford- 
shire, Leeds, Bury, Hertfordshire, Stockport, Essex. 

NEWCASTLE UPON TYNE - Salford, South Tyneside, Leeds, Gateshead, 
Sunderland, Sheffield, Liverpool, Middlesbrough, Hartlepool, Halton. 

NEWHAM - Hackney, Waltham Forest, Haringey, Brent, Southwark, Green- 
wich, Lewisham, Ealing, Islington, Birmingham. 

NORFOLK - Cornwall, Devon, Cumbria, Somerset, Lincolnshire, Suffolk, 
Shropshire, Herefordshire, Dorset, Derbyshire. 

NORTH EAST LINCOLNSHIRE - Doncaster, Redcar & Cleveland, Hartle- 
pool, Tameside, Telford & Wrekin, Halton, Rotherham, St Helens, Wigan, 
North Lincolnshire. 

NORTH LINCOLNSHIRE - Wigan, Doncaster, Derbyshire, Nottinghamshire, 
Rotherham, Cumbria, Telford & Wrekin, Dudley, Lincolnshire, Lancashire. 

NORTH SOMERSET - West Sussex, South Gloucestershire, Hampshire, 
Leicestershire, Essex, Worcestershire, Dorset, East Sussex, Gloucestershire, 
Poole. 

NORTH TYNESIDE - Darlington, Durham, Stockton on Tees, St Helens, 
Gateshead, Sunderland, Northumberland, Wakefield, Leeds, Wigan. 



the development of the Children’s Services Statistical Neighbour Benchmarking Model 



69 



NORTH YORKSHIRE - East Riding of Yorks, Warwickshire, Cheshire, 
Worcestershire, Rutland, West Berkshire, Staffordshire, Essex, Warrington, 
Hampshire. 

NORTHAMPTONSHIRE - Swindon, Kent, Staffordshire, Nottinghamshire, 
Warwickshire, Essex, Lancashire, Derbyshire, Worcestershire, Lincolnshire. 

NORTHUMBERLAND - North Tyneside, Stockton on Tees, Durham, Darling- 
ton, East Riding of Yorks, Derbyshire, Nottinghamshire, Staffordshire, 
Calderdale, Warrington. 

NOTTINGHAM CITY - Manchester, Barking, Sandwell, Hull, Southampton, 
Wolverhampton, Birmingham, Salford, Middlesbrough, Liverpool. 

NOTTINGHAMSHIRE - Derbyshire, Staffordshire, Lancashire, Cumbria, 
Northamptonshire, Swindon, Kent, Dudley, Wigan, Lincolnshire. 

OLDHAM - Rochdale, Kirklees, Tameside, Walsall, Bradford, Bolton, St 
Helens, Calderdale, Redcar & Cleveland, Halton. 

OXFORDSHIRE - Cambridgeshire, Bath & North East Somerset, West Berk- 
shire, Hertfordshire, Wiltshire, Hampshire, Gloucestershire, Bracknell Forest, 
Bedfordshire, B uckinghamshire. 

PETERBOROUGH CITY - City of Derby, Telford & Wrekin, Portsmouth, 
Coventry, Southampton, Sheffield, Plymouth, Southend, Walsall, Bolton. 

PLYMOUTH - Portsmouth, Torbay, Southampton, Telford & Wrekin, Peterbor- 
ough City, Southend, Bournemouth, Rotherham, Sheffield, Isle of Wight. 

POOLE - West Sussex, Dorset, North Somerset, East Sussex, Swindon, South 
Gloucestershire, Gloucestershire, Bournemouth, Kent, Essex. 

PORTSMOUTH - Plymouth, Southampton, Peterborough City, Southend, 
Bournemouth, Torbay, Bristol, Telford & Wrekin, Sheffield, City of Derby. 

READING - Bristol, Sutton, Brighton & Hove, Milton Keynes, Bromley, City 
of Derby, Bath & North East Somerset, Sheffield, Trafford, Hillingdon. 

REDBRIDGE - Hillingdon, Slough, Hounslow, Enfield, Merton, Barnet, Har- 
row, Ealing, Croydon, Sutton. 
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REDCAR & CLEVELAND - Doncaster, Rotherham, Tameside, Hartlepool, St 
Helens, Barnsley, Wigan, North East Lincolnshire, Halton, Stockton on Tees. 

RICHMOND UPON THAMES - Windsor & Maidenhead, Kingston on 
Thames, Surrey, Oxfordshire, Barnet, Wokingham, Bromley, Hertfordshire, 
Bracknell Forest, Buckinghamshire. 

ROCHDALE - Oldham, Kirklees, Middlesbrough, Bradford, Tameside, 
Bolton, St Helens, Walsall, Halton, Calderdale. 

ROTHERHAM - Doncaster, Redcar & Cleveland, Wigan, Barnsley, Tameside, 
Hartlepool, St Helens, Wakefield, Dudley, Telford & Wrekin. 

RUTLAND - North Yorkshire, West Berkshire, Wiltshire, Worcestershire, 
Cambridgeshire, East Riding of Yorks, Oxfordshire, Cheshire, Bucking- 
hamshire, Warwickshire. 

SALFORD - South Tyneside, Middlesbrough, Newcastle upon Tyne, Hartle- 
pool, Halton, Liverpool, Knowsley, Gateshead, Tameside, Sunderland. 

SANDWELL - Wolverhampton, Walsall, Coventry, City of Derby, Birming- 
ham, Barking, Nottingham City, Stoke on Trent, Peterborough City, 
Middlesbrough. 

SEFTON - Lancashire, Bury, Wirral, Stockton on Tees, Stockport, Notting- 
hamshire, Kent, Darlington, Southend, Swindon. 

SHEFFIELD - Leeds, City of Derby, Peterborough City, Portsmouth, 
Southampton, Plymouth, Dudley, Rotherham, Telford & Wrekin, Tameside. 

SHROPSHIRE - Devon, Somerset, Dorset, Suffolk, Gloucestershire, Wiltshire, 
Worcestershire, Herefordshire, Norfolk, Lincolnshire. 

SLOUGH - Hillingdon, Luton, Hounslow, Redbridge, Enfield, Coventry, Birm- 
ingham, Blackburn, Bradford, Wolverhampton. 

SOLIHULL - Warrington, Stockport, Cheshire, Trafford, Bedfordshire, Essex, 
Hertfordshire, Warwickshire, Bury, Hampshire. 

SOMERSET - Devon, Suffolk, Herefordshire, Shropshire, Cornwall, Dorset, 
Norfolk, Gloucestershire, East Sussex, Wiltshire. 
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SOUTH GLOUCESTERSHIRE - Hampshire, North Somerset, Leicestershire, 
West Sussex, Essex, Gloucestershire, Bedfordshire, Worcestershire, Poole, 
Kent. 

SOUTH TYNESIDE - Salford, Gateshead, Sunderland, Hartlepool, Newcastle 
upon Tyne, Halton, Middlesbrough, Barnsley, Knowsley, Liverpool. 

SOUTHAMPTON - Portsmouth, Bristol, Plymouth, Peterborough City, 
Sheffield, City of Derby, Telford & Wrekin, Coventry, Hull, Salford. 

SOUTHEND - Isle of Wight, Bournemouth, Torbay, Swindon, East Sussex, 
Portsmouth, Sefton, Kent, Medway, Telford & Wrekin. 

SOUTHWARK - Hackney, Lambeth, Haringey, Islington, Lewisham, Ham- 
mersmith & Fulham, Greenwich, Waltham Forest, Camden, Newham. 

ST HELENS - Stockton on Tees, Tameside, Darlington, Halton, Redcar & 
Cleveland, Doncaster, Wigan, Wakefield, Durham, Rotherham. 

STAFFORDSHIRE - Derbyshire, Nottinghamshire, Warwickshire, Northamp- 
tonshire, East Riding of Yorks, Worcestershire, Essex, Kent, Leicestershire, 
Lancashire. 

STOCKPORT - Solihull, Cheshire, Trafford, City of York, Warrington, Kent, 
Warwickshire, Essex, Bury, Lancashire. 

STOCKTON ON TEES - Darlington, St Helens, Bury, Calderdale, Wigan, Lan- 
cashire, North Tyneside, Sefton, Doncaster, Durham. 

STOKE ON T RE NT - Barnsley, Hartlepool, Redcar & Cleveland, Rotherham, 
Hull, Doncaster, Middlesbrough, Tameside, North East Lincolnshire, Wake- 
field. 

SUFFOLK - Somerset, Devon, Shropshire, Gloucestershire, Norfolk, Dorset, 
Cornwall, Herefordshire, Lincolnshire, Wiltshire. 

SUNDERLAND - Gateshead, Wakefield, Durham, Barnsley, Hartlepool, South 
Tyneside, St Helens, Halton, North Tyneside, Tameside. 

SURREY - Windsor & Maidenhead, Buckinghamshire, Bracknell Forest, Hert- 
fordshire, West Berkshire, Wokingham, Hampshire, Oxfordshire, 
Cambridgeshire, Bedfordshire. 
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SUTTON - Bromley, Bexley, Hillingdon, Hertfordshire, Reading, West Sussex, 
Bedfordshire, Swindon, Kent, Stockport. 

SWINDON - Kent, Northamptonshire, Essex, Medway, Nottinghamshire, Lan- 
cashire, Staffordshire, Poole, Stockport, West Sussex. 

TAMESIDE - St Helens, Redcar & Cleveland, Halton, Doncaster, Hartlepool, 
Rotherham, Stockton on Tees, Bolton, North East Lincolnshire, Wigan. 

TELFORD & WREKIN - Peterborough City, City of Derby, Thurrock, Lan- 
cashire, Doncaster, Dudley, Rotherham, Bolton, North East Lincolnshire, 
Medway. 

THURROCK - Medway, Dudley, Telford & Wrekin, Swindon, Havering, Lan- 
cashire, Kent, Wigan, Northamptonshire, Bolton. 

TORBAY - Isle of Wight, Southend, Bournemouth, Plymouth, Portsmouth, 
Blackpool, Telford & Wrekin, Cornwall, Poole, North Lincolnshire. 

TOWER HAMLETS - Camden, Newham, Islington, Westminster, Manchester, 
Hammersmith & Fulham, Hackney, Greenwich, Birmingham, Barking. 

TRAFFORD - Stockport, Solihull, Hertfordshire, Bromley, Bedfordshire, 
Cheshire, Warrington, Milton Keynes, City of York, Bury. 

WAKEFIELD - Durham, Sunderland, St Helens, Barnsley, Doncaster, Rother- 
ham, Wigan, Redcar & Cleveland, Stockton on Tees, Hartlepool. 

WALSALL - City of Derby, Bolton, Coventry, Rotherham, Peterborough City, 
Tameside, Telford & Wrekin, Doncaster, Rochdale, Dudley. 

WALTHAM FOREST - Croydon, Enfield, Haringey, Lewisham, Greenwich, 
Birmingham, Brent, Ealing, Merton, Luton. 

WANDSWORTH - Hammersmith & Fulham, Kensington & Chelsea, West- 
minster, Camden, Merton, Barnet, Islington, Greenwich, Reading, Enfield. 

WARRINGTON - Cheshire, Solihull, Stockport, Staffordshire, Warwickshire, 
City of York, Bury, East Riding of Yorks, Essex, Nottinghamshire. 

WARWICKSHIRE - Worcestershire, Cheshire, Staffordshire, Leicestershire, 
Essex, Bedfordshire, Kent, Northamptonshire, East Riding of Yorks, Hampshire. 
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WEST BERKSHIRE - Hampshire, Buckinghamshire, Bracknell Forest, Cam- 
bridgeshire, Oxfordshire, Bedfordshire, Hertfordshire, Wiltshire, Cheshire, 
Surrey. 

WEST SUSSEX - North Somerset, Hampshire, Essex, Gloucestershire, Dorset, 
South Gloucestershire, East Sussex, Worcestershire, Kent, Poole. 

WESTMINSTER - Camden, Kensington & Chelsea, Hammersmith & Fulham, 
Wandsworth, Islington, City of London, Greenwich, Haringey, Ealing, Barnet. 

WIGAN - Doncaster, Dudley, Rotherham, St Helens, Stockton on Tees, North 
Lincolnshire, Redcar & Cleveland, Wakefield, Nottinghamshire, Derbyshire. 

WILTSHIRE - Shropshire, Gloucestershire, Cambridgeshire, Worcestershire, 
Suffolk, Dorset, Hampshire, Oxfordshire, West Berkshire, Somerset. 

WINDSOR & MAIDENHEAD - Surrey, Buckinghamshire, Bracknell Forest, 
Hertfordshire, Wokingham, West Berkshire, Oxfordshire, Cambridgeshire, 
Hampshire, Bromley. 

WIRRAL - Sefton, Darlington, Stockton on Tees, Lancashire, Telford & 
Wrekin, Bury, St Helens, Halton, Tameside, Leeds. 

WOKINGHAM - Surrey, Windsor & Maidenhead, Buckinghamshire, Brack- 
nell Forest, West Berkshire, Hertfordshire, Hampshire, Oxfordshire, Cheshire, 
Bedfordshire. 

WOLVERHAMPTON - Sandwell, Walsall, Coventry, City of Derby, Birming- 
ham, Peterborough City, Nottingham City, Southampton, Sheffield, Bolton. 

WORCESTERSHIRE - Warwickshire, Shropshire, Dorset, Gloucestershire, 
Essex, Hampshire, Leicestershire, Kent, West Sussex, Wiltshire. 
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Appendix 8 Practitioner user-guide 

What are statistical neighbours? 



Statistical neighbour models provide one method for benchmarking progress. 
For each local authority (LA), these models designate a number of other LAs 
deemed to have similar characteristics. These designated LAs are known as sta- 
tistical neighbours. Any LA may compare its performance (as measured by 
various indicators) against its statistical neighbours to provide an initial guide as 
to whether their performance is above or below the level that might be expected. 

Currently, Ofsted provide a statistical neighbour model based on census data 
and the Institute of Public Finance comparator councils provide a model based 
on deprivation and demography data. But there is now a need for a new model 
based on the five Every Child Matters (ECM) outcomes, which embraces the 
key elements of the existing models and provides LAs and their partner agen- 
cies with a tool for assessing and comparing their performance with their 
statistical neighbours. Ideally, this new model would supersede the existing 
models within the context of children’s services and provide a single starting 
point for benchmarking performance. 



What is the tool? 

This tool has been specifically designed for children’s services authorities so 
that the statistical neighbours assigned to each LA are appropriate for compar- 
ing performance in terms of the five ECM outcomes. 

The National Foundation for Educational Research (NFER) was commissioned 
by DfES to develop this tool, to enable LAs to identify other LAs similar to 
themselves in terms of the socio-economic characteristics of their area. These 
comparative authorities are termed statistical neighbours. 
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How do I use the tool? 



The tool has been created in Excel. In order to identify the statistical neighbours 
for your LA, open the Excel Tool and go to the sheet labelled ‘Neighbour 
Finder’. 

1 Neighbour Finder Regional _ Metadata 

Select your LA from the box highlighted in yellow on the left hand side of the 
upper table. 




The lower table will automatically update and display your statistical neigh- 
bours, together with a measure of ‘closeness’ (explained below). 
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It is currently possible to compare the performance of your LA to the perform- 
ance of your statistical neighbours for a number of outcome measures (e.g. 
infant mortality rate). To select the outcome measure you wish to compare per- 
formance for, click on any of the boxes highlighted in yellow on the right hand 
side of the upper table (labelled Outcomes 1 to 5). 



Se/ecf Local Authority 




Indicator 1 


Indicator 2 


Indicator 3 


Indicator 4 


Indicator 5 


Barking & Dagenham 




Infant mortality rate: 
deaths up to 1 year per 
1 ,000 live births (2005) 
3.70 


U18 Conception Rate 
(2004) 
71.80 


5+A’-C inc. English & 
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3.99 
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The outcomes information displayed in the corresponding column of the lower 
table will automatically update enabling you to make comparisons. The last row 
of the upper table displays the average for the chosen outcome measures across 
all ten of your statistical neighbours. 



Average for statistical neighbours 


6.561 54.421 33.27 9.02| 3.9911 


It should be noted that the display of outcome information is intended as a sim- 
ple starting point for viewing performance to support benchmarking work. This 
is not to imply that further analysis of performance information such as compar- 
ison with targets or looking at trends over time is not required. 


The sheet labelled ‘Regional’ provides information on the similarity of your LA 


to all other LAs in the 


same government office region. This sheet is automati- 


cally updated to match the LA selected in the upper table of the first sheet. Each 
LA in the same region is listed in order of similarity. A measure of dissimilarity 
(explained below) is also shown. 


Neighbour Finder Regional 


Metadata 


Children's Services Statistical Neiqhbour Benchmarkina Tool 


Note: This sheet automatically updates based on selections in Neighbour Finder " 


Local Authority 


Region 


Barking & Dagenham 


London 


Other Local Authorities in Region 


"Closeness” 


Greenwich 


Close 


Enfield 


Somewhat close 


Hillingdon 


Somewhat close 


Bexley 


Not Close 


Sutton 


Not Close 


Hounslow 


Not Close 


Havering 


Not Close 


Waltham Forest 


Not Close 


Croydon 


Not Close 


Bromley 


Not Close 


Redbridge 


Not Close 


Bamct 


Not Close 


Merton 


Not Close 


Islington 


Not Close 


Ealing 


Not Close 


Hammersmith & Fulham 


Not Close 


Wands worth 


Not Close 


Kingston on Thames 


Not Close 


Camden 


Not Close 


Haringey 


Not Close 


Newham 


Not Close 


Lewisham 


Not Close 


Westminster 


Not Close 


Southwark 


Not Close 


Richmond upon Thames 


Not Close 


Harrow 


Not Close 


Kensington & Chelsea 


Not Close 


Hackney 


Not Close 


Brent 


Not Close 


Lambeth 


Not Close 


City of London 


Not Close 


Tower Hamlets 


Not Close 
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Which outcomes can be explored using these 
statistical neighbours? 



The tool defines a single set of statistical neighbours for each LA. These neigh- 
bours should be appropriate for use in benchmarking performance in most of the 
outcomes described in the ECM outcomes framework, but it does not provide a 
reasonable benchmark for the following outcomes: 

• Re-registrations on the Child Protection Register 

• % of children who have been looked after for more than two and a half years 
and of those, have been in the same placement for at least two years or placed 
for adoption 

• % of pupils permanently excluded from school. 

The reason for this is that the relationship between the levels of performance on 
these measures and the kind of socio-economic variables used to define statisti- 
cal neighbours is extremely weak. Because of this, these outcomes should be 
benchmarked against national averages rather than the performance of statistical 
neighbours. The reason for the weak relationship between these outcomes and 
the demographic information used to define statistical neighbours is probably 
due to the very small numbers of children involved in these outcomes. It is 
expected that the statistical neighbours that have been provided would be appro- 
priate for use in benchmarking almost any outcome based on a reasonably large 
number of children or young people. 

It should also be noted that since statistical neighbours are largely defined by the 
economic characteristics of local authorities, they should not be used to bench- 
mark outcomes concerned with levels of material deprivation, low income or 
unemployment. 

What does ‘Closeness’ signify? 

The third column in the lower table of the tool expresses the degree of similar- 
ity between your chosen LA and its statistical neighbours in terms of the 
measure defined above. The five levels of closeness are defined to match 
roughly the categorisation of similarity used by a previous set of statistical 
neighbours defined by Ofsted. Local authorities with a greater degree of simi- 
larity to your own LA might be expected to provide the best comparison in 
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terms of performance. However, in order to avoid undue emphasis on the per- 
formance of a few statistical neighbours, it is advised that generally speaking a 
comparison with all ten of the designated statistical neighbours will provide the 
most robust benchmark for performance measures. 

How does this model compare to Ofsted’s 
statistical neighbours? 

Forty-six percent of the 1 ,500 statistical neighbours that have been defined in 
total were also defined as statistical neighbours for the same local authorities by 
the model devised by Ofsted in 1998. In other words for most local authorities 
about half of their statistical neighbours will have changed from those defined 
by the Ofsted model. This degree of change is to be expected as the data used to 
define neighbours is somewhat different in each model. 



Where can I find further details? 

Further details on the process of variable selection and weighting are available 
in the full report on the development process. 
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